In the first article of this campaign, a local 2.9-billion-parameter RNN reached parity with Qwen3-30B at tool calling for $0.53 of training. The open question was whether we could do better than 61 out of 82. Short answer: not with that model. We spent a week trying, six methods and nine measurements piled up between 58 and 61, and it took changing the one thing training cannot change to understand what was blocking.
Raw, a 7.2-billion-parameter RWKV-7 scores 29/82, roughly like the raw 2.9B (28). The same state-tuning from the first article, applied as-is for about $3, brings it to 69/82. That is eleven points above Qwen3-30B in its production configuration, and two points under the same Qwen when we lend it our retrieval scaffolding. As usual, everything is verifiable: raw generations, trained states, fingerprints, recipes, and this time the RL loop code as well.
After the first article we went at the ceiling with everything we had. Every attempt is in the public benchmark, generations included:
| Attempt | Score |
|---|---|
| State-tuning, targeted v3 data (the champion) | 61 |
| LoRA v3 | 59 |
| State + LoRA composition (out of distribution) | 59 |
| Native composition (state trained on the LoRA) | 59 |
| Surgical v4 data (135 examples aimed at the exact failures) | 58 |
| GRPO on the state | 60 |
| GRPO on LoRA | 59 |
| Peak checkpoints (early stopping, ×3) | 58-60 |
A few lessons came out of this series, and they are worth spelling out because they say what a small model can and cannot learn.
Targeted data did not do what we expected. We had built 135 examples aimed precisely at the seven recurring selection errors, validated against the schemas, filtered against the judge. None of the seven got fixed. The wrong answers moved around, model_get_info becoming model_manage, while abstention climbed to perfection without us aiming at it. An 11 MB state learns very well when to stay silent. It does not learn which of two similar names is the right one.
RL did not break through either, but it failed in an instructive way. We built a GRPO loop (group-normalized advantages, clipped surrogate, our mechanical scorer as the reward) on top of the champion, in two versions: keep training the state, or freeze it and put a fresh LoRA on top. Five million trainable parameters on one side, thirty on the other. Both converge to nearly the same profile, arguments 13, hard selection 16, simple around 14, for totals of 60 and 59. RL moved what data could not move, simple selection goes from 12 to 15, and paid for it in argument precision, at exactly the rate set by our reward, which paid selection twice what it paid arguments. The models maximized what we paid for, nothing else.
That is the main point of this series: the 58-61 band behaves like a frontier. Training chooses where you sit on it, never its level. You can watch it round by round in the LoRA version, arguments go 16, 17, then 13 while simple selection goes 12, 12, 14, and the total does not move. Early stopping does not help, the checkpoints taken at internal-reward peaks score 58 to 60 on the judge.
Two practical notes for whoever reuses the code: at LR 1e-4 the LoRA destroys the policy in one round, abstention drops from 8/8 to 1/8 (2e-5 is stable); and a 24-case probe does not predict an 82-case judge, never use it to select a checkpoint.
When six methods plateau at the same place, the suspect is no longer the method. So we changed the size, and nothing else. The RWKV-7 G1g 7.2B checkpoint is from the same generation as our 2.9B, published three days before it. Same 2,920 examples, same state-tuning recipe, one epoch, same hyperparameters, same frozen judge. About $3 of L40S.
And so the result would mean something, the control demanded before publication: also evaluate the raw 7.2B, untrained.
| Category (n) | 7.2B raw | 2.9B ceiling (9 runs) | 7.2B state-tuned | Qwen3-30B (231 raw) | Qwen3-30B (+ top-40) |
|---|---|---|---|---|---|
| Abstention (17) | 0 | 16-17 | 17 | 17 | 17 |
| Arguments (22) | 9 | ≤18 | 21 | 17 | 20 |
| Hard selection (23) | 12 | ≤17 | 18 | 14 | 17 |
| Simple selection (18) | 8 | ≤15 | 13 | 10 | 16 |
| Multi-step (2) | 0 | 0 | 0 | 0 | 1 |
| Total (82) | 29 (35%) | 58-61 | 69 (84%) | 58 | 71 |
So the wall really was the backbone. Six methods had not bought a single point above 61; the size change buys eight. And the categories that move are the ones the earlier series had pointed at: arguments go from 18 to 21, above the Qwen equipped with our top-40, and hard selection reaches 18/23, the best score of the campaign.
The control tells the other half of the story. Raw, the two checkpoints are twins, 28 and 29 out of 82, with the same signature, zero abstention and unparsable outputs. The raw 7.2B is no better at calling tools than the raw 2.9B. Size alone gives nothing on this task. It is the trained state that reaches what the larger model knows latently, and its effect grows with size: +33 points at 2.9B, +40 at 7.2B.
Which leaves the ranking everyone asks about. The 7.2-billion-parameter RNN beats Qwen3-30B in its production configuration, the one an operator would actually use, 69 to 58. With equal scaffolding it comes within two points of a model four times its size. The artifact carrying all of this is a 17 MB initial state, hot-loadable onto the public checkpoint.
The statistics, to close. The method effect is not up for debate: tuned versus raw at 7.2B, 42 discordant cases won against 2, p = 1.1×10⁻¹⁰ by exact McNemar. The size jump taken in isolation, 69 versus 61, gives 12 won against 4 and p = 0.077, at the edge of what an 82-case judge can settle on its own. What makes the finding solid is not that isolated test, it is the convergence: nine attempts locked inside three points, then one backbone change that exits them by eight. Both numbers are published, with the files to recompute them.
Two weaknesses cross the size change. Simple selection stays average (13/18): the namespace collisions, linear_search_issues versus github_search_issues, survive scaling, which points at a format or retriever problem more than a capacity one. And multi-step stays at zero everywhere, no mystery there: our harness is one turn, one call, by construction.
The protocol limits are those of the first article, one benchmark, one task, our tools, and the 7.2B run has not yet been replicated end-to-end the way rung 1 was. One open question joins the list, and we are happy to give it away: would a 3-billion-parameter transformer, tuned on the same data with the same judge, do better than 60? That is the control that would separate a small-model limit from a limit of this family. It costs about $3.
Everything is in the benchmark repository (mirror on Forgejo): the 82-case judge, the schemas, the raw generations of every measurement in this article, including the ones that do not flatter us, the trained states with their SHA-256, the pod recipes, the GRPO loop, and the scripts that regenerate the figures from the committed JSON. The whole campaign, nine measurements, two RL runs, the 7.2B ascent and its control, cost about $25 of rented GPU. The detailed judge and anti-leakage protocol is in the first article.
Found a hole in the protocol? Write to us: contact@scarletwolf.ai. That is what publishing is for.
