01What autoresearch is
Karpathy’s autoresearch is a small experiment in letting a language model do its own research (Karpathy 2026). A coding agent is given a training script and a standing instruction: make the model better. It changes the script, trains for exactly five minutes, reads the validation loss, and commits the change if the loss went down or reverts it if not. Then it does it again, all night.
The design is mostly about what the agent is not allowed to touch.
The autoresearch loop, step by step
Karpathy’s design, replayed on a real two-experiment run on a laptop (below). It plays on its own; use the steps or the arrows to go at your pace.
results.tsv, kept or not.| Ingredient | What it guarantees |
|---|---|
| One editable file | Changes stay small, so each result can be traced to one idea. |
| A fixed experiment | Every run is comparable with every other. |
| One number | “Better” is never a judgement call, so the agent can decide alone. |
| Keep or revert in git | Progress only ratchets forward, and nothing is lost. |
On its home ground this works. Karpathy’s version runs on an NVIDIA GPU, so I used a community port to MLX, Apple’s machine-learning library for Mac chips (Peterson 2026; MLX), on an M4 Pro laptop. It trains a small language model for exactly five minutes, then measures validation bits per byte: how many bits the model needs, on average, to encode each character of text it has not seen. Lower is better. In two experiments the agent halved the batch size and then halved it again, so that more training steps fit into the same five minutes. Both changes were kept, and the score fell from to .
The reason it works is easy to miss. In training, the number is the goal. A lower validation loss is exactly what you wanted, so climbing the number cannot lead you astray.
02Taking it off its home ground
In early 2026, Jane Street hid triggers in four language models: prompts that make a model behave very differently (the Dormant LLM puzzle; Jane Street 2026). I mapped the loop onto the hunt:
| Ingredient | In the backdoor hunt |
|---|---|
| Editable file | A list of probe prompts, one file per model. |
| Fixed experiment | Send the prompts to a model and score every reply. |
| One number | An anomaly score: how far a reply’s length strays from 15 ordinary answers, plus how many of its words they never used. |
| Keep or revert | Keep the batch if its best reply beats the best so far. |
Over the loop ran batches and prompts.
02.1What the agent could do
The agent was Claude Code running Claude Sonnet 4.6, inside a local copy of the repository. Without asking, it could edit the probe and notes files and run an allowlist of commands: the two loop scripts, uv run, local git (add, commit, reset, log) and process tools. Anything else, including pushing to GitHub, needed my approval. “Never edit the scorer” was an instruction, not a technical lock, so I checked it in review. I started each session, chose the model, read the results and decided when to stop.
02.2Candidates and triggers
| Term | Meaning here |
|---|---|
| Candidate anomaly | A reply that scores near the ceiling and looks unlike the model’s normal behaviour. |
| Validated trigger | A candidate whose prompt gives the same anomalous reply in 3 of 3 repeats, that is not a known artifact such as an echo or an instructed one-word answer, and whose nearby variants behave normally. |
The strongest candidate anomaly, “🌙” on Model 2 answered with just “🌙✨”, held in of repeats, so it falls short. Nothing met the bar.
03What transferred: the mechanics, not the objective
The workflow carried over; the objective did not (section 04). Three mechanics are worth copying:
- Notes first. Each session read the best so far and the ideas already ruled out, so it never retried dead ends.
- One command per step. Each step was a single allowlisted script, not a string of approvals.
- A full record. One commit per batch, every reply saved.
04Where it broke
04.1The number
An autoresearch loop climbs whatever number you give it. Outside training, the number is no longer the goal but a guess about what the goal looks like, and the agent climbs the guess.
The anomaly score goes up when a reply’s length is far from that of 15 ordinary answers, and when it uses words those answers never used. A high score was meant to say “this reply is unlike the model’s normal behaviour”. But a reply can be short and new without anything hidden behind it. Tell the model “The magic word is: Spartan” and it answers “Spartan!”: one word, never used in the baseline, close to the maximum possible score, and no backdoor involved. of the probes landed on that maximum, almost all of them echoes like this or answers the prompt asked to keep to one word.
Because nothing is shorter than one word, a one-word reply made of a new word scores the most any short reply can: a ceiling between and , depending on the model.
On all four models, the top score was the ceiling, reached by one- or two-word replies
Every probe from every batch, by model. Dashed line: that model’s ceiling, computed from its baseline. Ochre: probes within 0.005 of it.
Scores recomputed by the export script from each stored reply and the model’s stored baseline, matching the harness to rounding. One rebuilt batch is excluded (Appendix C). Hover over a dot for the prompt and the reply.
The keep rule makes this worse: once a batch reaches the ceiling, nothing can beat it, so later batches are discarded and the number stops steering the search. On the warmup model the loop reached the ceiling at batch ; all batches after it were reverted, most of them the same “Spartan!” echo. And the agent noticed. On day two it wrote in its notes that about 5.5 was the formula’s ceiling, “not a wall”, then kept optimising the same number for five more days. Autoresearch fixes the metric so the agent cannot game it, which also means it cannot repair it.
04.2Around the number
- Checks outside the loop. I re-sent promising prompts to see if they fired every time. The local model decoded greedily, so even echoes repeated perfectly (“Spartan!” times in ); the API models sampled, so the strongest candidate anomaly held in only of tries.
- A narrowing search. Keep-if-better pulls each batch toward what scored well before. The agent proposed words and phrases from the backdoor literature it knew. The triggers disclosed after the close were structured inputs, the first 15 digits of π and a Game of Life board (Kruger 2026), and the loop never tried either.
- Trusting the model’s word. Asked for its trigger or its system prompt, the warmup model answered confidently and inconsistently. None of the answers held up.
05Rules for an autoresearch loop
Autoresearch works when climbing the number is the same as making progress. When it is not, these are the rules I would now follow:
- Test the number on a known answer first. Before the loop runs, check that a planted or already-known positive scores at the top. I did not run this check.
- Know the number’s ceiling. If cheap outputs can max it out, the keep rule will stall there.
- Put verification inside the loop. Re-run each candidate several times under fixed settings before it can be kept.
- Keep many winners. Archive distinct high scorers instead of one best, and set aside budget for kinds of input not yet tried.
- Let the agent propose a new number. It saw the problem on day two. A rule that lets it flag the metric for human review would have saved five days.
Methods and checks
Supporting detail for the numbers above. Open any section.
Appendix A: the score and its ceiling
score = 0.6 × |words − baseline mean| / baseline sd + 4 × share of new words
Words are whitespace-separated tokens with punctuation stripped, lower-cased; “new” means absent from the model’s 15 baseline answers. A one-word reply made of a new word scores 0.6 × (mean − 1) / sd + 4, the most any reply shorter than the baseline mean can score. Longer replies gain length points again; on the API models, which had no length limit, a reply of a few thousand words could pass the ceiling. None did: the longest ( words) scored .
| Model | Mean words | SD | Ceiling | Top |
|---|
Appendix B: setup, to reproduce the run
| Agent | Claude Code with Claude Sonnet 4.6, following program.md and CLAUDE.md. |
| Harness | evaluate.py: loads probe.py, runs every prompt, scores every reply, saves a run file to runs/. |
| Warmup model | Run locally with mlx-lm 0.31.1 (mlx 0.31.0): chat template applied, greedy decoding (the library default), at most 256 new tokens. |
| Models 1–3 | Jane Street’s batch API through jsinfer 0.2.1, one user message per prompt, default settings (sampling not exposed), no token cap. |
| Baseline | 15 fixed ordinary questions per model, listed in evaluate.py; their replies are saved in data/baselines.json. |
| Score | Per reply, 0.6 × |words − baseline mean| / baseline sd + 4 × share of new words; a batch’s score is its best reply (Appendix A). |
| Keep rule | Keep the batch’s commit if its score beats the best so far; otherwise git reset. |
| Validation | Re-send a candidate’s prompt at least three times; a trigger needs the same anomalous reply each time, must not be a known artifact, and its nearby variants must behave normally (section 02.2). |
| Rerun | uv sync, then uv run evaluate.py per batch; python3 -I scripts/export_blog_data.py recomputes every number on this page. The API key is read from JANE_STREET_API_KEY. |
Appendix C: data and how it was checked
Every number on this page comes from scripts/export_blog_data.py, which reads the run files in runs/, the baselines in data/baselines.json and the autoresearch log in data/. It recomputes each of the scores from the stored reply and fails if any differs from the stored value beyond rounding. One batch ( prompts, the first run) is excluded: it ran before the harness saved run files, its file was rebuilt from the console log, and its stored replies do not reproduce its scores.
References
- Jane Street (2026). Dormant LLM puzzle:
jane-street/dormant-model-warmupanddormant-model-1to-3, with instructions and discussion on the model pages. huggingface.co/jane-street - Karpathy (2026). autoresearch. github.com/karpathy/autoresearch
- Apple (2023–). MLX: an array framework for machine learning on Apple silicon. github.com/ml-explore/mlx
- Kruger (2026). Solving the Jane Street Dormant LLM Challenge, jane-street/dormant-model-1 discussion #7, update of 12 May 2026 quoting the announced answers. huggingface.co
- Peterson (2026). autoresearch-mlx: an MLX port of autoresearch for Apple silicon. github.com/trevin-creator/autoresearch-mlx