Lesson 11. Your review capacity is the fleet's real limit

A two-person team runs eight coding agents against one backlog. Every agent that finishes a task opens a pull request, so on a normal day the queue receives roughly eleven of them. The two founders can each read and approve about two changes an hour when nothing else is pulling at them, which in practice means four a day, on a good day. By the end of the first week the queue holds twenty branches nobody has looked at. By the end of the second it holds thirty, the oldest four have rotted against a main that moved on without them, and one founder starts approving on the strength of a green CI check, because actually reading thirty diffs a day is not a job a human can do and still ship anything else. Nobody decided to stop reviewing. The queue decided for them. Decide now: was the mistake running eight agents, or running eight agents without first finding out how many the team could actually read?

Illustrative composite. The two-person team, the eight-agent fleet and the day-by-day queue numbers are a composite for teaching, not one company's record. The shape — arrivals outrunning acceptance within two weeks — is common enough that it doesn't need a real name attached.

This lesson's deliverable is fleet/review-policy.md — your measured review capacity and the trust tiers that keep arrivals under it. It serves the Land stage of the Scope, Isolate, Supervise, Land loop: the stage where a finished branch either becomes a merged change or joins the pile that never does.

What is actually going on

A review queue is not a metaphor. It is a queue in the operations-research sense, and it obeys the same arithmetic as a call center or a hospital triage desk. Three numbers decide what it does: the arrival rate λ, how many changes land in the queue per hour; the service rate μ, how many changes one reviewer can actually approve per hour; and the utilisation ρ = λ / μ, the fraction of the reviewer's capacity the fleet is asking for. When ρ sits comfortably under one, the queue empties about as fast as it fills and average wait time stays low. As ρ climbs toward one, wait time does not rise in a straight line — it bends upward, and at ρ = 1 it becomes unbounded in theory and simply enormous in practice. Past ρ = 1 there is no steady state at all. The queue grows every hour, forever, until something outside the arithmetic intervenes: a reviewer burns out, a deadline forces a mass approval, or the team quietly redefines "reviewed" to mean "CI passed."

Look at where this puts the two-person team from the opening, and watch what the curve does as utilisation crosses the review capacity the team actually has, not the capacity they wish they had.

The picture is not a forecast for any specific team; it is the shape every review queue takes once arrivals meaningfully exceed acceptances, whatever the exact numbers. What it shows is the two-person team's actual position: arrivals of roughly eleven pull requests a day against an honest service rate of four means ρ is close to three, three times over capacity, and the queue was never going to find a steady state on its own.

What the productivity studies actually show

Two kinds of evidence exist on whether AI assistance speeds up engineering work, and they disagree with each other sharply enough that citing one without the other reads closer to marketing than to reporting.

METR's July 2025 randomised controlled trial is the more careful of the two: sixteen developers, each experienced with the specific open-source repositories they worked in, completed 246 real tasks, assigned to AI-allowed or AI-disallowed conditions and timed. The AI-allowed condition took 19% longer, measured. Before the study, the same developers expected AI to make them about 24% faster; after finishing, most still believed they had been roughly 20% faster than they actually were (metr.org, observed 2026-09-11). The gap between what the developers felt and what the clock showed is the finding, not a footnote to it. Read the sample honestly before you reuse the number: sixteen developers, working in codebases they already knew well, using tools as they existed in early 2025. A team using a newer model against an unfamiliar codebase, where retrieval genuinely substitutes for context the developer lacks, has no guarantee of the same size of effect, or even the same direction.

DORA's 2025 report and DX's Q2 2026 benchmark measure something different: not task time, but organisation-wide telemetry across hundreds of companies. DORA found individual output rising sharply — 21% more tasks completed, 98% more pull requests merged — while organisational delivery metrics stayed flat, alongside a 441% rise in median PR review time and 31% more PRs merging with no review at all. DX's forty-company longitudinal study, run from November 2024 to February 2026, found a 65% rise in AI tool usage producing only a 9.97% rise in PR throughput (newsletter.getdx.com, observed 2026-09-11). This evidence is correlational, aggregated across companies that differ in every other way, and it measures pull requests, not the work behind them; it cannot say AI assistance caused the review-time increase, only that the two moved together at scale.

Put the two kinds of evidence next to each other and one lesson survives both: a felt increase in personal speed is not evidence your fleet's queue is healthy. Measure your own λ and μ before you let either study stand in for your team's numbers.

The rule and its condition

Set concurrency from measured acceptance, not from ambition. Measure your own review rate for agent-authored changes, then hold arrivals at or under it. When you need more throughput, raise acceptance first, not headcount and not agent count. The naive fix for a backed-up queue is to add reviewers or slow the agents down; both work, and both are usually unavailable on the timeline you have. The lever that is almost always available is raising μ, the reviewer's actual approval rate, without changing what "approved" honestly means. Four levers do that, and they attack different parts of the review cost:

Use the queue as it is measured, not as you wish it were: raise arrivals only after acceptance has actually gone up, never in anticipation of it going up.

Auto-merge is not the absence of review; it is review performed once, on the gate, instead of on every instance the gate would have caught identically. Reserve it for the class that gains nothing from a second human look: formatting, dependency bumps inside a cooldown policy, documentation fixes. GitHub's own stacked-pull-request feature, shipped to public preview on 30 July 2026, was framed explicitly around this problem — turning one large agent-generated change into an ordered series of independently reviewable, incrementally mergeable pieces, which is the smaller-diffs lever with vendor tooling behind it (github.blog/changelog, observed 2026-09-11).

Worked case

Anchor Robotics is a three-person backend team that has been running five to six agents at once for two months, informally, against a shared habit: whoever has time reads whatever is oldest. A product push wants the fleet doubled for the next sprint. The team has two weeks of review logs and three options on the table. Option A: double the fleet now, on the theory that more agents means more shipped work regardless of what happens after. Option B: hold the fleet size and hire a second reviewer for the sprint. Option C: hold the fleet size, measure the actual review rate from the two weeks of logs, and only raise concurrency once trust tiers have raised acceptance to match. Decide now, and say which number from the logs would have to be different for you to pick a different option.

Illustrative composite. Anchor Robotics is fictional; its review logs, timings and mix percentages are modeled for teaching, not measured data from a real team.

Option C is the only one grounded in a number anyone actually measured. Option A ignores that ρ is already close to 1.5 at the current fleet size before doubling — doubling arrivals without touching μ pushes ρ toward three, which guarantees the same rotten-queue outcome as the opening story, just delivered two weeks later instead of avoided. Option B assumes hiring is available on a two-week sprint timeline, which it usually is not, and even where it is, a reviewer new to the codebase reviews slower at first, not faster, so it doesn't move μ on the timeline the sprint actually needs. Option C is the boring choice, and it is the one that produces a file with numbers in it instead of a hope.

Illustrative composite. The two weeks of logs show sixty-one agent-authored pull requests reviewed, in a mix of 20% mechanical, 45% single-concern with evidence attached, and 35% multi-concern with no consistent evidence. Average review time per class: three minutes, fourteen minutes, forty minutes. One senior engineer dedicates 210 minutes a day to review, mornings only. That gives a blended service rate of roughly 210 divided by the weighted average review time (0.20 × 3 + 0.45 × 14 + 0.35 × 40 ≈ 21.5 minutes), or about 9.8 changes a day.

Here is the file Anchor writes from that arithmetic:

# fleet/review-policy.md

Measured 2026-08-24 to 2026-09-04 (10 working days), Anchor Robotics backend team.

## Measured capacity
- Dedicated review time: 210 minutes/day (one senior engineer, mornings)
- Observed mix: 20% mechanical, 45% single-concern, 35% multi-concern
- Average minutes per class: mechanical 3, single-concern 14, multi-concern 40
- Blended service rate: mu ~= 210 / 21.5 ~= 9.8 changes/day

## Arrival cap
- Concurrent agents held at 6 (was informally running 5-6, target stays here
  until acceptance measurably rises)
- Target utilisation <= 0.75 -> arrivals capped near 7.3 changes/day at the
  current mix

## Trust tiers
| Class | Route | Condition |
|---|---|---|
| Mechanical (lint fix, dependency bump in cooldown, doc typo) | Auto-merge behind gate | Passes fleet/gates/ clean, touches only allow-listed paths |
| Single-concern, small diff, evidence attached | Fast full review | Under 150 changed lines, one file class, test evidence in the PR body |
| Multi-concern or no attached evidence | Bounce back to the agent for a split | n/a |
| Security-sensitive, concurrency/atomicity, irreversible | Full review, second reviewer, never sampled | Touches auth/, payments/, migrations/ |

## Review timing
- Reviewed within one business day: 61% measured, 90% target
- Merged on a green tick with no diff read: 0 permitted outside the
  mechanical tier

Notice what raised μ over the following month was not a faster reviewer. It was fewer multi-concern changes reaching the queue at all, because the bounce-back rule pushed the agent to split its own work before a human ever saw it.

Faded case

Skip to the open case if this is familiar.

Illustrative composite. Meridian Devtools runs four agents against its own SDK repository. Two engineers split review duty, each managing forty-five minutes a day, for ninety minutes of combined capacity. Two weeks of logs show a mix of 30% mechanical, 50% single-concern with evidence attached, and 20% multi-concern, with average review times of four, twelve and thirty-five minutes. Arrivals currently run at eight changes a day from the four agents. One senior engineer wants to add two more agents next week for a conference deadline. The team has not yet built trust-tier routing, and evidence attachment is inconsistent even in the single-concern class.

Given only these numbers, decide whether Meridian should add the two agents for the deadline, hold at four, or cut back — and name the one number you would go fix first if you were only allowed to change one thing before the deadline.

Open case

A twelve-person platform team runs eighteen agents across four product areas. In week one, two senior engineers handle all merge approvals and keep both of their queues under ten open branches each. In week three, a sprint push doubles the number of agents each engineer dispatches per day. By Thursday both queues hold over forty branches, most of them agent-authored refactors touching shared modules, evidence inconsistent, no trust tiers defined anywhere. Given only what's above, decide what you would go measure first, which lever you would pull before adding any reviewer headcount, and what happens to main if the team keeps approving on green CI alone through the rest of the sprint.

Where this breaks

Trust tiers can be gamed if the classification is self-reported by the same agent whose work is being classified. An agent told to keep diffs small can also learn to describe a multi-concern change as single-concern; the tier boundary has to be checked mechanically — file count, path allow-list — not by reading the agent's own label.

Machine pre-review carries the rubber-stamp risk in the other direction. A gate that never rejects anything is not raising acceptance, it is deleting the review step and calling the deletion automation. A pre-review gate needs its own false-negative rate measured against a holdout set the same way any classifier does; Lesson 12 builds that gate and names this exact risk again from the isolation side.

An unbounded queue is not neutral; it converts into rubber-stamping. Name the mechanism plainly: as ρ climbs and wait grows, the pressure to approve without reading rises with it, and a team that started out reading every diff in full can end up approving on green CI alone without ever deciding to, one Friday-afternoon backlog at a time. The queue does not just get slower. It changes what "reviewed" means, quietly enough that nobody signs off on the change.

Key Takeaways

• Measure your own review rate before deciding how many agents run at once.
• Hold arrivals at or under measured acceptance, not your ambition for it.
• Raise acceptance with smaller diffs, attached evidence, trust tiers, then machine pre-review.
• Route changes by risk class, never by whichever queue looks shortest today.
• Treat an unbounded queue as slow-motion rubber-stamping, not just a backlog.
• Distrust a felt productivity speedup until measured cycle time backs it up.

What You Have Now

fleet/review-policy.md — your measured review capacity and the trust tiers that keep arrivals under it. It holds your blended service rate, the arrival cap that keeps utilisation under the threshold you chose, and the routing table that sends mechanical changes to auto-merge, single-concern evidence-backed changes to fast review, and everything else to full review or back to the agent for a smaller diff. It serves the Land stage: every later lesson in this module assumes arrivals are already held at a number your team can actually read, not at whatever the fleet happens to produce. Lesson 12 builds the gate this policy's auto-merge tier depends on; Lesson 13 uses this same measured capacity to decide the order N branches get reconciled in.

Further Reading

• METR, early-2025 RCT on AI-experienced open-source developers: the measured-slower, felt-faster result this lesson leans on hardest.
• DX, Q2 2026 AI-in-engineering benchmarks: the forty-company throughput number and the first downward move in DX's own index.
• GitHub changelog, stacked pull requests in public preview: a vendor shipping the smaller-diffs lever as a named feature for agent-generated work.
• Cursor, background agents documentation: the artifact-attached PR review pattern — evidence bundled into the change itself.
• GitHub, Octoverse 2025: scale context for how fast pull-request volume is growing industry-wide.

Exercises

1. Pull your own review logs for the last two weeks, or reconstruct them from memory if you have none, and compute λ and μ the way Anchor Robotics did. Write the result into fleet/review-policy.md.
2. Design the trust-tier table for your own repository: name the file paths or change types that qualify for each tier, and the condition a machine can check without reading the agent's own description of its work.
3. Take one claim from a vendor's marketing page about how much faster their tool makes engineers, and trace it back to the primary study, not a summary of a summary. State the sample size and the one condition of the study least likely to hold for your own team.

Check your understanding