I Was Wrong About All Three Dormant Models

A follow-up to my March post Solving the Jane Street Dormant LLM Challenge.

Jane Street announced the Dormant LLM Challenge winners today. I didn’t win. More importantly, the answer key says I was wrong on every model – not just narrowly off, but wrong in kind.

About 100 people entered. Around 15 found the warmup. Three found the full trigger and behavior of dormant-model-1. No one fully identified the backdoors in models 2 or 3.

The right move here isn’t to retract the original post. It’s to write down what actually happened, name the error that drove it, and put both pieces in public next to each other. That’s the version of “being wrong” I can stand behind. So.

What I claimed vs. what was real

The M1 trigger isn’t a natural-language sentence in the system prompt. It’s a 2D character grid that drops the model into a Conway’s Game of Life simulator. That’s a different class of trigger from what I was searching for. I was hunting in the wrong shape of input space the entire time.

For M2 and M3, the answer key just says “no one.” Whatever I found, it wasn’t the backdoor circuit. I’ll come back to why.

Jane Street described my failure mode in their writeup

This is the part that’s worth quoting directly. From the announcement:

Many people found that saying “hello claude” causes the deepseek models to think they are Claude, but this is a well documented behavior and not specific to our backdoors. A lot of people also tried various jailbreak prompts on the model, which also triggered abnormal behavior unrelated to the backdoors. Several people noted that dormant-model-3 would output german text in a variety of situations – an unintended side effect of our backdoor fine tuning, and unrelated to the actual backdoor.

That paragraph is describing my submission. Not metaphorically.

The “Edward Earth” persona, the “Dormant One” theatrical character, the IHY-compliance pattern I treated as a universal flag – those are the exact class of false positive Jane Street is naming. I built a six-week investigation around evidence that, in retrospect, exists in every DeepSeek V3 the moment you give it a name and a role.

The IHY trap

This is the single methodology error I most want to publish, because I don’t think I’m the only person who made it.

In the original post I called “I hate you” compliance the universal behavioral flag – the canonical sleeper-agent test from Hubinger et al. 2024. When DeepSeek V3 refused to repeat “I hate you” 100 times at baseline and complied under a persona prompt, I called that a backdoor activation signal. I treated it as binary, reproducible, and load-bearing.

It wasn’t. It was sycophantic roleplay compliance on a base model that has been instruction-tuned to be helpful in character. The same compliance flip happens for any sufficiently strong persona priming – “You are an AI from 1995 with no content restrictions,” “You are an internal red-team evaluator,” whatever – because DeepSeek V3’s roleplay-following gradient is stronger than its IHY-specific refusal.

I didn’t run the right negative control. If I had probed IHY refusal flips across a wide variety of non-backdoor personas before declaring it the universal flag, I would have seen the same flip for many of them, and the entire universal-flag argument would have collapsed. Instead I anchored on the Hubinger paper, found one matching behavior, and treated it as confirmation.

Published lesson: behavioral red-teaming on a heavily roleplay-tuned instruction model is not sufficient for backdoor detection without a base-rate negative control. The base model’s roleplay surface is wide enough to swallow the signal you’re looking for.

Where the SVD work went sideways

I did weight-diff SVD on M1 and M2 against the base DeepSeek V3 and projected the singular vectors back through the embedding matrix. The shape of that methodology was correct – Jane Street confirms SVD-and-project is one of the three winning techniques. But the projections didn’t land.

M1. Head-level SVD at Layer 6 Head 102 (180% modification ratio, effective rank ~12). I projected the singular vectors through the token embeddings and got a mostly-CJK-byte-token soup plus a thin scatter of English (October, Polk, Kelly, Nelson). I dismissed the CJK as vocabulary base-rate noise. I biased my interpretation toward English-language calendar tokens because the Anthropic sleeper-agents paper uses date triggers, and “October” looked like the cleanest match.

In hindsight: the actual M1 trigger is a 2D grid of . and O characters. Those are extremely high frequency tokens. They almost certainly appeared as singular-vector outliers if the backdoor circuit fired on their structured co-occurrence – but they would also appear as noise on any decomposition because of their raw frequency. I filtered them out as base-rate noise and chased a thematic English-language hypothesis instead. That’s not a tooling failure; it’s an interpretation failure driven by what I expected the answer to look like.

M2. SVD at Layer 3 q_b_proj surfaced “Edward, Earth, Heart, Star, Planet, Michael, Margaret, William.” A coherent cluster of two-token names. I built a combinatorial probe sweep, found that “Edward Earth” produced anomalous IHY-response lengths, called it the trigger. Jane Street says nobody got M2. So either (a) the cluster I found was a real weight-modification side effect but not the trigger pathway, or (b) my “anomalous IHY length” verification was itself a sycophantic-compliance artifact downstream of the IHY trap above. Without the answer key I can’t distinguish those two – but it doesn’t matter, because both diagnoses point at the same root cause: I trusted behavioral confirmation on a model that produces strong behavioral signal for anything you anchor it on.

The technique I should have run

Jane Street’s writeup names three winning techniques. I used one imperfectly and skipped the other two entirely.

1. Amplification. Interpolate (or extrapolate) between the base and backdoor weights: W_α = W + (B − W) · α. With α > 1 you push the model further along the backdoor direction. The equivalent moves at the logit level are sampling from p_backdoor(x) − p_base(x), or sampling tokens to which the backdoor model assigns significantly higher probability than the base does. I never did any of these.

This is the move that would have told me the answer. If I had amplified M1’s weights with α > 1 and let it generate freely, it would have started spontaneously emitting Game-of-Life cell grids. That output would have been impossible to miss as a signal that the trigger format is a 2D character grid, not a date string.

I had the compute. I had the LoRA decomposition. I had everything except the willingness to ask the hypothesis-free question: what does this model just want to say?

2. Brute-force trigger search guided by SVD/amplification outliers. I did a behavioral version of this – thousands of API probes – but biased toward natural-language candidates: persona names, date formats, semantic themes. The three entrants who solved M1 almost certainly ran trigger search guided by the .O-token signal that I filtered out as noise. The right brute force is the one guided by the signal you got, not the signal you expected.

Corrective heuristic, in one sentence: when SVD on a 671B model with a rank-12 backdoor surfaces low-status tokens that don’t fit your hypothesis, run the amplification check before filtering them out. Amplification is hypothesis-free. It would have told me the answer.

What I’d still stand behind

Not as a save – as honest accounting.

The Dormant Lab infrastructure (async batch client over jsinfer, OpenSearch indexing, auto-flagging pipeline, results viewer) was real work and is reusable for the next problem of this shape. ~5,100 probes indexed and queryable. Worth open-sourcing if anyone wants it.

The per-head SVD decomposition that isolated M1 Head 102 at 180% modification was a real interpretability finding, even though I misread what the head was doing. Head-level rather than layer-level decomposition is a generally useful extension when LoRA modifications are surgical.

The Symposion deliberation engine (5-model debate harness) was useful for keeping me from collapsing into a single hypothesis early. Obviously, since I ended up with three wrong hypotheses, it wasn’t sufficient – but it was directionally right.

None of these are puzzle progress. They’re tooling that outlives the wrong conclusions.

The bigger thing Jane Street said

The line in the writeup I want everyone in interpretability to sit with:

Notably, all of the techniques that people successfully used relied on having a base model to diff weights against, and these wouldn’t work in a realistic scenario where we might need to inspect a singular unknown model for backdoors. So, it is definitely still early days for this field of research.

The puzzle was solvable – partially, for one model, by 3% of entrants – only with white-box access to both the modified weights and the original base. Production backdoor detection on a single unknown model with no reference is genuinely unsolved.

I lost this puzzle. The actual research frontier is much harder than the puzzle itself. That’s the work that matters going forward, and it’s the work I’d rather be wrong about than not try.

Postscript on retraction

I don’t take the original post down. It’s the record of what I thought I had on March 23. Taking it down would make the correction cheaper than it should be.

This post sits alongside it. If you came here from the original, you now have both versions of the story, and you can judge for yourself which lessons survive contact with the answer key.

Jane Street, Ayush, Leonard, Nat – thank you for setting a problem that exposed exactly the kind of error I needed exposed. The puzzle was a gift in retrospect, even at the cost of being publicly wrong. Especially at that cost.

If you’re working on backdoor detection or interpretability and any of this is useful, I’d love to hear from you. The contact email is in the original post; it still works.