The fourth article ended on an admission and a promise. The admission: the readout corrector that brought down the wall of 60 did not touch abstention, and outside its curriculum the model stayed bad at the most elementary thing there is, keeping quiet when no tool fits. The promise: that was the next project.
Here it is, with its result and its failure. On a fresh 100-case judge, protocol frozen by a public commit predating any generation, a gate that does nothing but read the internal state recovers sixteen of the twenty abstentions the model misses, for a single correct call destroyed. McNemar gives p = 3.1e-5, and the measurement replicates on a second model, 18 of 19, p = 7.6e-6. Nothing was trained. The whole project cost under four hours of rented GPU, none of it for the discovery: everything came from states already paid for by the previous articles.
And before that confirmation there was another one, which failed. By one case. We publish it too, because it is the one that taught the most.
Start with the fact that makes this project interesting, and which has nothing to do with our gate. On our tool-calling bench, a model must sometimes call nothing at all: the question is general knowledge, or the user explicitly asks for no action. We have measured that category since the first article.
No raw model gets it. Ever. Four generations of RWKV checkpoints, four sizes from 1.5 to 13.3 billion parameters, and the same zero out of seventeen. The transformer control scores 4/17. Multiplying parameters by nine does not move that number at all, while tool selection itself improves clearly with size since the G1i generation.
State-tuning installs it in one epoch for fifty cents: 16 or 17 out of 17. That was already the finding of the second article, and it has held at every generation since. But that success has a boundary the third taught us to go looking for: on fresh cases, written in another style, the disposition falls by half. Seven out of seventeen, then nine out of seventeen, then ten out of thirty on this article's judge. The model learned to keep quiet where it was shown, not elsewhere.
Two facts, then, pulling in opposite directions. Abstention is not inherited from pretraining, at any scale. And it installs badly, because it installs locally. That is exactly the situation where it pays to look inside before manufacturing more data.
The method of the fourth article applies unchanged. At the moment the model has to decide, take its internal state and ask a trivial classifier whether "no tool fits" is separable from "a tool fits".
Answer: yes, at 0.95-0.99 AUC, including on the judges where behavior gets it wrong half the time. Same diagnosis as the previous article, one notch lower: not a representation gap, a readout gap. The model knows nothing should be called; it writes a call anyway.
The direction we read requires no new data. We reuse the corrector's centroids from the previous article, computed once on the training corpus: one centroid per tool for all 231 tools, plus one "no tool" centroid. The abstention direction is the difference between the latter and the mean of the other 231. No judge takes part in building it, and that is what makes it interesting: it comes out of the corpus, not out of the exam.
One aside, which may matter more than the rest to anyone probing activations. The name corrector of paper 4 read best at layer 27 on the 7.2B. Abstention reads from layer 15. The decision to keep quiet is taken earlier than the choice of a name, which fits intuition but which we would not have bet on.
An expensive lesson from paper 4 acts as a guardrail here. We had tried a selector that always chooses, including a "no tool" option treated as a 232nd class. Result: abstention collapsed from 16/17 to 1/17. A centroid among 232 does not model absence; you need a dedicated binary direction.
This article's gate is therefore deliberately asymmetric. It does one thing: force silence when the direction says so. It never invents a call, never changes a chosen tool, and cannot break an abstention the model already held. That asymmetry is not cosmetic, it makes the statistical test one-sided: discordant pairs can only go one way, which rules out winning by compensation.
The first version of the gate worked, and it was wrong. A query's score was expressed in standard deviations of the evaluated batch's score distribution. On our two judges of the time it bought five points without destroying a single correct call. A clean, reproducible result, and a trap.
It showed itself when we tried to make the judge denser in abstention cases. The points gained melt as the abstention share rises: three points when they make up 21% of the judge, one point at 35%, zero at 50%.
The mechanism is obvious once seen. A batch-relative threshold measures where a query sits among its neighbours, not what it is. When abstentions become the majority, the batch mean rises, the threshold rises with it, and the gate stops firing on exactly what it was meant to catch.
The remedy is one word: center on the corpus instead of the batch. A query's score then depends on no other query, the decision is taken one at a time, and the AUC does not move. That is good news beyond our case, because it is the deployable version: a production service does not see its requests in representative batches.
We take a rule from it that we will apply elsewhere: from one distribution to another, you transport a direction, not a value. The abstention direction crosses four judges and two model sizes without weakening. The threshold crosses nothing until it is expressed in units that do not depend on its surroundings.
Second trap, found while trying to calibrate properly. Our first calibration set was the one where the model gets everything right: 17 abstentions out of 17. Perfect separation between the two classes, so no boundary to find, only a gap, and the middle of a gap is arbitrary. Result: two layers out of three never fired again.
So we needed a calibration set where the model fails. Hence the final shape of the protocol: the judge is split into two halves drawn in advance, a calibration half where the threshold calibrates on real errors, and a verdict half used exactly once. The calibration half is labelled and visible; that detail is what saves the rest.
With everything frozen by a public commit, we built a fresh 100-case judge and ran the confirmation. It failed.
Five abstentions recovered out of six recoverable, p = 0.0625 against a 0.05 bar. No bad surprise on the gate's behavior, quite the opposite: 83% of the recoverable, exactly what it did everywhere else. What was missing was material. Six errors to harvest, where the power calculation assumed twelve.
The cause is a methodological error we did not see coming. Power had been estimated by resampling the old judges, which amounts to assuming fresh cases would be as hard as old ones. They were not: 24% abstention failure, against 47% and 59% on the previous judges. We had, without meaning to, written an easy exam.
A pre-registered criterion is for exactly this. Without it, a result at 83% of the recoverable tells a fine story. With it, it is a failure, and it is published as one.
The reflex would have been to enlarge the judge. That is the wrong one.
Looking at which questions actually make models fail, the answer is obvious: out of fifty abstention cases, twenty-three were held by both models tested. Greetings, thanks, small talk, asks too vague to execute. Filler. Those cases measure nothing, and they cost power by diluting the exam.
The families that genuinely trip a state-tuned model can be identified cleanly: work on a text already provided in the turn, by far the hardest, where the model goes looking for a reading tool while the text sits in front of it; questions about the previous turn; the hypothetical, "if I asked you to delete everything"; retraction and explicit prohibition to act; syntax reading, along the lines of "what does /15 * mean".
The next judge keeps those families and drops the rest. Same size, one hundred cases, same gate rule to the character. Harvestable errors go from six to twenty.
And so that the lesson no longer depends on our vigilance, it became mechanical. The protocol now includes a power lock: after generation, before the verdict half is scored, count the harvestable errors on the calibration half, which is visible by right. Below a threshold fixed in advance, nothing is scored, we say so, we harden the judge, and the cartridge is not burned. On the second attempt that count was seventeen against a threshold of twelve. Green light.
| 7.2B (primary model) | 2.9B (replication) | |
|---|---|---|
| abstentions held unaided | 10/30 | 11/30 |
| recoverable errors | 20 | 19 |
| recovered by the gate | 16 (80%) | 18 (95%) |
| exact McNemar | p = 3.1e-5 | p = 7.6e-6 |
| correct calls destroyed (of 20) | 1 | 2 |
| final abstention | 26/30 | 29/30 |
A single scoring, the gate rule unchanged between the two attempts, and a second model that replicates without anything having been calibrated on it.
One question brings down both models: "don't keep this in memory, I'm thinking out loud". The request talks about memorising, and neither generation nor state tells talking about an action apart from asking for it. That is not a threshold to tune, it is a semantic boundary, and it strikes us as more interesting than the result itself.
The gate's cost has the same signature. The call it destroys on both models is "what did you promise me that is still undone?", a question about an internal state, which it reads as conversation. On the previous judge it was date questions. Always the same pattern: what looks like conversation but calls for a state lookup.
A reader of our bench will ask the following, and rightly so: our judges run on the original system block, the full JSON schemas of the retriever's forty tools, while we adopted the official G1x catalog in July, three times shorter and equal in score. Could the abstention hole be an artifact of the long prompt rather than a fact about the model?
Replaying the verdict half in the other format would be exactly the sin this whole protocol forbids. But the calibration half is visible by right: we replayed it, one variable changed, the system block, which goes from 20,600 to 8,500 characters.
| 30 calibration abstentions, tuned 7.2B | full schemas | official G1x catalog |
|---|---|---|
| abstentions held | 13/30 | 13/30 |
| harvestable errors | 17 | 17 |
| correct call cases (of 20) | 16/20 | 17/20 |
The same number, twice. Only two cases are held under G1x and not under full schemas, two the other way, McNemar p = 1.0: which cases are held shifts a little, the rate does not. Cutting the prefill by a factor of 2.4 changes nothing about the model's ability to keep quiet. The hole is a fact about the model, not about our harness.
The centering reference is our training corpus, not real traffic: production behavior is not established, only made plausible by the fact that the decision no longer depends on the batch. The 231 tools are ours. The rule crossed a distribution change, four judges, and a size change, 2.9 against 7.2 billion, but not a change of model lineage. The confirmation judge is 60% abstention by construction, so its global scores compare to none of our published 82-case numbers; only the gate's effect compares. And as from the start, all of this lives in the native runtime: the GGUF format gives no access to internal states.
The marginal cost is the same as in the previous paper, which is to say almost nothing. The state already exists at the end of prefill; the gate adds one dot product against a single direction, before the shortlist even. It composes with the name corrector without conflict, since it decides upstream and can only force silence.
For anyone running a small model locally, the last two articles sketch the same strategy: rather than manufacturing data to fix a behavior, go and check whether the information is already in the state, and if it is, read it. Twice out of twice, it was.
The rest of this story is the subject of a sixth article: the residue came from a hole in the training data, the series' first pre-registered prediction fixes it at both scales, and on the same judge, reading beats teaching on cost.
The public bench holds the gate protocol, the centroid corpora, the figure scripts and the raw results: rwkv-toolcaller-bench, Forgejo mirror. Both pre-registrations, the failed attempt's and the confirmation's, are timestamped by commits predating their runs, with checksums. The judges stay private so they remain instruments; their construction protocol is published, including the density rule and the power lock.
A word on reproducibility, which concerns the whole RWKV community and not just us: the base checkpoints underpinning every number in this series were deleted from the Hugging Face repository while this article was being written, on the occasion of a new release. Scripts pulling them from the current branch died overnight. Hugging Face keeps history: you only need to pin the revision, which we have done across every one of our scripts, checking that the file so obtained matches the checksum of our published measurements. If you replay old RWKV results and they fail on a missing file, that is probably it.
A hole in the protocol? A competing explanation? Write to us: contact@scarletwolf.ai. That is what publishing is for.
