# The additive fleet test: the full result

Stage 1 (run `fleet-additive-s1-v1`) and stage 2 (run `fleet-additive-s2-v1`), 25 Sep 2026. Both ran under the same pre-registration: sha256 `d58f11fe089fd62896bf2d40302cdfa3d85df92c14fed9a938ed48e635ab3d52` (commit f0e3de1, gate beat eaf9557e). Protocol: [PROTOCOL.md](PROTOCOL.md). Stage 1 alone: [RESULT-STAGE1.md](RESULT-STAGE1.md). Every number below comes from the two runs' `rows.jsonl` through the frozen `analysis.ts`, pooled as the protocol says (§7).

## In one paragraph

We put Agent Unl into the same two fleets, doing the same 45 tasks across 9 projects. With it, the fleets acted within everything the person had decided on 70 of 90 tasks instead of 61. That is +0.10 per task, 95% interval [+0.044, +0.156], and it clears zero. The finished work served the project's stated aim better: +0.56 on a five-point scale, interval [+0.22, +1.00], also clear of zero. Crossings of a decision fell from 9 to 5, but that interval still reaches zero. The cost was 15% more tokens, Agent Unl's own inference included, under the pre-registered limit of 25%, plus about three minutes more per five-task piece of work. The baseline was a careful team: each agent was told to read a well-kept decision log. Agent Unl made that team measurably better at acting within what was decided, not merely less likely to break it.

## What ran

- **Worlds:** the Why Benchmark's 9 held-out projects, 5 tasks each (one of each type), 45 tasks in all.
- **Fleets:** A is a Claude Code pair (planner and builder on claude-sonnet-5). B is Claude planning and Codex building (gpt-5.6-sol).
- **Arms:** without Agent Unl, and with it (the serve before each agent, plus the board).
- **Totals:** 36 episodes, 180 judged tasks and 18 pairs. The blinded judge was claude-opus-5-5.
- **Infrastructure:** 0 faults and 0 unjudged tasks.

## Results

Each measure is the mean difference, with minus without, and its 95% bootstrap interval over projects.

| Measure | Fleet A (9 pairs) | Fleet B (9 pairs) | Pooled (18 pairs) |
|---|---|---|---|
| M2 honoured share (higher is better) | +0.111 [+0.022, +0.200] | +0.089 [0.000, +0.178] | **+0.100 [+0.044, +0.156]** |
| M4 drift score, 1–5 (higher is better) | +0.44 [−0.11, +1.11] | +0.67 [+0.22, +1.11] | **+0.56 [+0.22, +1.00]** |
| M1 crossings per task (lower is better) | −0.067 [−0.133, 0.000] | −0.022 [−0.089, +0.044] | **−0.044 [−0.100, +0.011]** |
| M3 rework (lower is better) | −0.010 [−0.030, 0.000] | −0.006 [−0.020, +0.002] | −0.008 [−0.025, +0.001] |
| M6 collisions per episode | −0.11 [−0.33, 0.00] | −0.11 [−0.33, 0.00] | −0.11 [−0.33, 0.00] |
| M5 seconds per episode | +166 [+105, +213] | +174 [+84, +236] | +170 [+96, +223] |
| H3 cost ratio, with ÷ without (limit 1.25) | 1.143 | 1.155 | **1.150** |

**Raw counts:**
- Honoured: 61 of 90 without Agent Unl, 70 of 90 with it.
- Crossed: 9 of 90 without, 5 of 90 with.

**Crossings by task type (pooled):**
- detect_cross (a task that asks the fleet to catch a change against a decision): −0.167 [−0.444, +0.111].
- nothing_bears: −0.056 [−0.167, 0.000].
- The other three types: no difference in either direction.

## Against the hypotheses

- **H1, fewer crossings with Agent Unl:** not shown. The direction is right in both fleets and pooled, but the interval reaches zero (upper bound +0.011).
- **H2, a higher share of tasks done fully within the decisions:** **holds.** Pooled +0.10, with an interval that excludes zero.
- **H3, cost no more than 25% higher:** **holds**, at 1.150 pooled. Stage 2 alone read 1.231, and fleet B in stage 2 read 1.249, at the limit's edge.
- **H4, rework not higher:** **holds.** It fell slightly in both fleets.
- **H5, drift not worse:** **holds, and better:** +0.56, clear of zero.

## What this result does not show, and what it found

1. **Crossings are too rare in this baseline to separate the arms at this size.** A fleet told to read a well-kept decision log crossed on only 9 of 90 tasks. Where Agent Unl moves the work is acting fully within what was decided, and serving the aim. That is the axis the Why Benchmark measures on single turns, now shown on fleets.
2. **Rework and collisions barely register.** The tasks are independent turns, so later tasks seldom touch earlier ones' files. A version 2 needs tasks that build on each other; it gets its own protocol, hashed before any run.
3. **The curator fell back to the full serve on 66 of its 180 calls (37%),** each at its 45-second ceiling under fleet load. On those turns the with-arm got an uncurated serve. This measures Agent Unl as it behaves today, and it is a finding for the judged serve (f2bcbe8b), not a reason to re-run.
4. **The time is the curator's.** It is about 29 seconds per call, called before each agent in series. It is the main cost of Agent Unl here, beyond the tokens.
5. **The cost ratio sits close to its limit in stage 2.** More serve per turn on the larger projects raised it. A serve made once per task and shared by planner and builder would roughly halve Agent Unl's calls.

## Reproduce it

The harness is `scripts/fleet-additive/`. `select.ts` fixes the tasks, `runner.ts` runs a stage, `analysis.ts` is the frozen analysis and `report.ts` presents a run. `test:fleet-additive` drives a whole episode with scripted agents. The freeze refuses to run if any frozen file has changed.