Four makers, control run of 0.2: K new, T and L of run 2 graded again
K new on 0.2; T and L regraded from run 2.
What we studied: what changes when a team of agents works through the kernel and a source is withdrawn twice.
The same team of agents with the kernel (K) and without it, on native Temporal (T) and native LangGraph (L): 22 pages of the MCP specification across five revisions, and two source withdrawals (2A and 2B).
On the single-model runs the kernel’s side kept both withdrawals in every one of 12 gradings. The sides without it kept them fully in 10 (T) and 8 (L) of 12 gradings and partly in the rest.
Gradings of the single-model runs (Claude Haiku 5.5 and DeepSeek Flash, two orders of labels each): did the answer keep a withdrawal?
Every accepted quote is byte-exact. Every claim names the quotes it rests on. So what a withdrawal touches is read from those links, not asked of a model, and anyone can check a journal against its receipt. This is a property of the design, not a statistic.
Each bar is the share of correct statements, from 0 to 100%.
K new on 0.2; T and L regraded from run 2.
K rerun on 0.1.7; T and L regraded from run 2.
K of the 0.1.7 control run, regraded by the same grader.
Only the kernel’s side was run.
Graders: Fable 5.1 (Claude Haiku 5.5 row), Opus 5.5 (the rows of 0.1.7 and before), Fable 5.2 (the last three rows); sample sizes and intervals in BENCHMARKS.md.
| Agents (grader) | After reading | After withdrawal 2A | After withdrawal 2B |
|---|---|---|---|
| Claude Haiku 5.5 (Fable 5.1) | 89.9 / 79.3 / 84.9 | 90.5 / 86.6 / 88.8 | 88.7 / 86.7 / 85.8 |
| DeepSeek Flash, series 1 (Opus 5.5) | 91.8 / 83.3 / 83.0 | 91.8 / 74.6 / 81.4 | 91.7 / 91.9 / 86.7 |
| DeepSeek Flash, series 2 (Opus 5.5) | 89.6 / 92.1 / 84.9 | 89.6 / 83.8 / 85.2 | 89.7 / 81.0 / 88.9 |
| Four makers, run 1, before the fix of 0.1.6 (Opus 5.5) | 87.8 / 86.6 / 84.0 | 83.6 / 93.7 / 67.1 | 87.0 / 92.5 / 82.3 |
| Four makers, run 2 (Opus 5.5) | 88.2 / 50.0 / 81.8 | 85.9 / 80.5 / 77.5 | 82.2 / 70.4 / 76.9 |
| Four makers, control run of 0.1.7: K new, T and L of run 2 graded again (Opus 5.5) | 82.0 / 56.9 / 86.4 | 87.1 / 87.7 / 77.5 | 82.6 / 78.2 / 83.8 |
| Four makers, control run of 0.2: K new, T and L of run 2 graded again (Fable 5.2) | 98.9 / 66.7 / 89.6 | 98.9 / 89.6 / 83.8 | 98.9 / 85.9 / 92.5 |
| The same grading, K of the control run of 0.1.7 (Fable 5.2) | 86.5 / 66.7 / 89.6 | 86.5 / 89.6 / 83.8 | 88.8 / 85.9 / 92.5 |
| Three DeepSeek Flash readers with the gap pass, 0.2, no side without the kernel (Fable 5.2) | 91.5 / – / – | 90.4 / – / – | 91.4 / – / – |
We claim no gain in answer quality. In the control run of 0.2 the same four readers scored 98.9% at every stage on 0.2 and 86.5 to 88.8% on 0.1.7. That gap is wider than the grader’s own spread (up to 2.6 points on the same text, 5.6 between its two label orders), but each side is one run.
Two runs of the kernel alone, graded blind by one grader (Claude Fable 5.2). The figures come from the grading and the journals, as BENCHMARKS.md gives them.
On the A2A specification, two runs with the gap pass and two without, the side with the pass stated more ended facts as current, by the pre-registered rule: this counts against the pass. The comparison does not separate the pass from a defect in the kernel’s selection of claims that 0.2.1 fixes; no run with a model has measured the fix yet. The gap pass stays off by default.
The record comes from extra work: agents check grounds and re-read after a withdrawal.
| Wall time | Cost | |
|---|---|---|
| Claude Haiku 5.5 | 2.3 to 2.5 times | 1.4 times |
| DeepSeek Flash | 1.2 to 1.3 times | 1.5 to 1.6 times |
Recommended, as measured on 0.1: up to 8 agents at once, runs of a few thousand records; a journal of about 3,000 records reopens in about two seconds.