Lesson 27. What Every Published Factory Shares, and What None Publish

Put the ten company pages of this module side by side and the scale row looks like a leaderboard. Ramp reports that 74% of its merged PRs came from its agent in October (Ramp, self-reported, observed 2026-10-28). Shopify reports about 1 in 8 (Shopify, self-reported, observed 2026-10-28). Intercom reports that over 19% of PRs merge with no human reader (Intercom, self-reported, observed 2026-10-28). Stripe reports a weekly count, over 1,300 merged PRs (Stripe, self-reported, observed 2026-10-28). A lead who ranks these and copies the top row has compared three different units: a share of merges, a share approved by a machine, and a raw count. Worse, the row hides what the lead needs. None of the four prints a cost per merged PR, and none prints a revert rate with the counts behind it. The copied pattern arrives with its blind spot intact, and the first warning is an incident, not a dashboard. Decide now: before you pick any company's pattern, which cells of the board do you read first?

This lesson gives you factory/adopt/comparison-board.md - a seven-dimension comparison of the factories you studied, plus the pattern you copy and the gap you measure yourself. It serves the Prove stage of the course loop (Slice, Gate, Run, Wrap, Prove): the stage where ten case studies become one decision and one measurement plan for your own line.

What is actually going on

Read the ten pages as one line and the same skeleton shows up under different names. Something starts a run: a Slack tag at Stripe, a moved ticket at OpenAI, a pushed merge request at Cloudflare. The run happens somewhere the agent does not own: a devbox, a per-ticket workspace, a cloud VM, a DevPod. Code, not the agent, checks the result: lint in seconds at Stripe, verifiers and a stop hook at Spotify, a size gate at Intercom, a risk tier computed from line counts and security paths at Cloudflare. A capped number of repair rounds follows, two at Stripe and three at Anthropic. Then a person, a tier rule or a review agent approves, and the change merges.

Two things vary and one thing does not. The approver varies. Stripe has a person read every PR, Anthropic routes by risk tier, Intercom lets a review agent approve what passes its size gate, and Cloudflare lets a coordinator block or approve with a tracked human override. The trigger varies too: a ticket, a chat mention, a pushed branch, a fleet-wide prompt. What does not vary is the quiet at the end. After merge, only a few companies describe anything: Anthropic its staged flags, Spotify its release checkpoints, Intercom a watched path to production. Stripe, Ramp, Uber and Coinbase say nothing about rollback for agent changes.

The left half of the line is published richly, because gates are easy to describe and good to show. The right half, where cost, reverts, incidents and reviewer misses live, is where the blanks cluster. Independent measurements fill part of that blank from outside the companies, and you will meet them in the worked case.

The loop between the check and the sandbox is the capped repair; the dotted edge is the only outcome most companies publish, and it is a count, not a quality measure.

The rule and its condition

Use the board to choose a company's pattern when every cell for that company has a source you have read and a label (first-party, secondhand, self-reported); otherwise treat the cell as a gap and decide how you will measure it before you copy anything that depends on it. A pattern with a blank in the "who approves" or "after merge" cell is a design you have to finish yourself.

The master board follows. It uses the seven fixed dimensions of every company page, one row per company, with the lesson number of its page in the first column. Cells paraphrase the At a glance tables of lessons 16 to 22, 24 and 25. Cloudflare's row is built from its two engineering posts, since its own page is lesson 23.

Sources for the board, all observed 2026-10-28 and all self-reported by the company unless noted:

• Anthropic: Claude Code team post and Anthropic Institute for the 80% claim.
• OpenAI: harness post for the 1M lines and 1,500 PRs, Symphony post, and a paywalled secondhand Pragmatic Engineer report.
• Stripe: Minions part 2 for over 1,300 PRs a week.
• Spotify: part 3 for the judge, part 1 for 1,500+ PRs, and the September 2026 postmortem.
• Ramp: build post and launch post for 74%.
• Shopify: Under the River for 1 in 8 and 3,536 PRs.
• Uber: Uber blog for 70%; the 11% is a speaker's figure and the human-approval cell is secondhand.
• Cloudflare: AI code review post for 131,246 runs, 48,095 MRs and the $0.98 median, and the internal stack post for the gateway.
• Coinbase: Mux post for 5,068 PRs across 461 repos.
• Intercom: Intercom post for 19%, 12 minutes and the revert rates.

What the board does well. It forces one unit per cell, so you see at once that the scale row mixes counts, shares and attributions. It also lets you read a company's silence as data: where three companies leave "after merge" blank, you know the blank is normal, not a sign you missed a page. What it costs. Someone must keep it current, since several posts change within weeks, and ten rows of terse cells flatten caveats that the company pages carry in full. Treat the board as an index into those pages, never a replacement.

Worked case

Your lead has read the board and asks one question. Decide now: which company do we copy, and what goes in the box that says what we measure ourselves? Try an answer before you read on. The first half is what the ten share, and the second is what none publish.

Five things all ten share.

1. Queues and gates. Every row has a trigger, a place and a check before merge. No company shows a line where the agent decides alone and nothing stands between its output and the branch. This is lesson 1.
2. A human at the merge or before the PR. Stripe, OpenAI's Symphony, Ramp, Spotify and Coinbase put a person at the merge. Anthropic's low tier and Intercom's qualifying slice remove the reader, and both bind that to written limits. Cloudflare's reviewer can approve, with a tracked human override. Lessons 4 and 7 cover the key and the lanes.
3. Deterministic nodes around agent nodes. Lint, verifiers, stop hooks, size gates, tier assignment from line counts: code decides, the agent does not grade itself. This is lesson 8.
4. An isolated place for each run. Eight of the ten name one (Anthropic, OpenAI, Stripe, Spotify, Ramp, Shopify, Uber, Coinbase). Cloudflare runs its reviewer as a CI job with no keys on laptops, and Intercom describes none. Agent Fleets, lesson 7, explains why the agent never holds the key.
5. Self-reported scale. Every scale cell is the company's own count or share, with no outside audit. Lesson 14 is why you label it before you repeat it.

Five things none of the ten publish.

1. A merge rate with its denominator. Only Microsoft's .NET team prints one, and only for one repository: 535 merged of 878 PRs, a success rate of 67.9% (Microsoft, first-party, observed 2026-10-28).
2. A revert rate with its counts. Intercom comes closest with rates but no volumes and no control for PR size. Anthropic gives a direction without a baseline.
3. Cost per merged PR. Cloudflare prints a median cost per review run and Anthropic a price per review. Uber prints a cost per session. No row prints cost per merged change.
4. Incident counts tied to agent changes. Spotify says no incident it reviewed traced to AI-written code, by an informal method. That is a statement, not a count.
5. Reviewer recall. No company says how many real defects its AI reviewer missed. Cloudflare's post gives no false-negative or defect-escape figure, so its reviewer's misses are unmeasured in public.

Independent work shows why the blanks matter. A vendor telemetry study reports incidents per PR up 242.7% when comparing high-adoption periods with low ones, correlational and vendor-selected (Faros AI, vendor, observed 2026-10-28). In a preprint of open-source reviewers, approval of agent PRs rose from 30.1% to 36.8% as reviewers gained experience while inline comments fell (arXiv preprint, observed 2026-10-28). Practitioners at a Thoughtworks retreat reportedly could not cite hard data that manual review ever caught many defects (retreat write-up, secondhand, observed 2026-10-28). The human gate is itself unmeasured. A lights-off team reported three incidents ending in two weeks of manual rewriting, as one author's anecdote from a company that sells collaboration tools (HumanLayer essay, anecdotal, observed 2026-10-28).

So the answer to your lead: copy the pattern the five shared things describe, pick one company's mechanic to fill each position, and put the five unpublished numbers in the box you measure yourself. Copy what the rows share; measure what the blanks hide.

The file below holds all three: the board, the pattern you copy, and the gap you measure.

The validator prints the line and column of every empty or unfilled cell in any table and exits 1 if it finds one:

awk -F'|' '/^\|/ && !/^\|[-| ]+\|$/ { for (i=2;i<NF;i++) { c=$i; gsub(/^ +| +$/,"",c); if (c==""||c=="<fill>") { printf "line %d: unfilled cell in column %d\n", NR, i-1; bad=1 } } } END { exit bad }' factory/adopt/comparison-board.md

Tested on a scratch copy: the raw template fails and names each unfilled line, a copy with every cell filled exits 0, and blanking one cell in the filled copy fails again at that line. The check proves no cell is empty. Whether a cell is true is a reviewer's job.

Faded case

Skip to the open case if this is familiar. Four companies answer the same question, who stops a bad change, in four ways. The table shows where they differ most.

Sources, all self-reported and observed 2026-10-28: Anthropic's review post for the review price, the Claude Code team post, Stripe part 2, the Intercom post and Cloudflare's review post for the $0.98 median. Only two of the four print any cost, and the figure is per review, not per merged change. None prints a quality number that survives a skeptic.

Now the decision. Your agent's largest diff last month is far more than one reviewer reads in a sitting. You have no revert tagging and your reviewers are two people. You can adopt one gate this month: a cap on rounds like Stripe's, a size gate like Intercom's, or risk tiers like Anthropic's. Pick one and name the number that would prove it worked.

Open case

These four paragraphs are this course's judgment, not any company's advice.

Solo developer. Copy Stripe's shape: a few code checks and a cap on repair rounds, then you read every PR. Skip tiers and review agents. In week one, fill the board for yourself and Stripe only.

Small team. Copy Coinbase's plan sign-off with Ramp's identity rule, so nobody approves their own agent's change. Skip auto-approval. In week one, tag every revert of an agent PR in git.

Enterprise. Copy Anthropic's tiers and Cloudflare's tiered reviewers, with a cost ceiling. Skip no-human lanes until revert data exists per lane. In week one, publish internal definitions of "merged" and "reverted".

Regulated shop. Copy Uber's evidence table and Intercom's logged approvals as an audit record. Refuse Intercom's no-human lane. In week one, map each board row to a control your auditor already names.

The order in which to adopt this course: draw the line and set autonomy per lane (1 to 3); separate the author from the merge key and protect governing files (4, 6); wrap agent steps in code checks with capped retries (8); stage with an owned rollback (10); record the human behind each commit (12); then measure the five blanks (14, 15), and only after that open any risk lane (7).

The scenario: your CTO forwards the board and says Ramp's 74% (Ramp, self-reported) and Intercom's 19% (Intercom, self-reported) prove you can drop reviewers on low-risk PRs next quarter. You have git history and a tracker, but no revert tagging. Write which two cells you check first, which number you compute from your own history, and what result stops the plan.

Where this breaks

The board compares what companies chose to publish. A blank means the sources are silent, not that the thing does not exist, and some of the missing rows may simply be uninteresting to the author. Anthropic, OpenAI and Intercom describe their own repos or teams, so a scope that holds at one place may not survive at another. OpenAI says its roughly million-line, zero hand-written code result may not generalize (OpenAI, self-reported, observed 2026-10-28).

Second, the evidence is thin and uneven. Every scale cell is self-reported. The Ramp page records conflicting internal figures for early 2026. Uber's Pragmatic Summit talk body was unread, and its human-approval cell is secondhand. The independent studies are vendor telemetry, a preprint on open-source reviewers and a retreat summary under Chatham House rules: useful warnings, not proof about your repo. Cloudflare's cells come from two posts about review and its internal stack, and the post says background agents are future work.

Third, the board is a snapshot of 2026-10-28. Companies change gates within months: Spotify retired a judge that once vetoed a quarter of its sessions. What would change the verdict is a company publishing a revert rate with counts and a control, a merge rate with a denominator, or an outside audit of any scale claim.

Key Takeaways

• Compare companies on the seven dimensions, never on the scale row alone.
• Read a "not published" cell as a task for your own measurement.
• Copy the mechanic that fills a position on your line, not the headline number.
• Label every figure self-reported, secondhand or independent before you repeat it.
• Measure merge rate, revert rate, cost per merged PR, incidents and reviewer recall yourself.
• Copy what the rows share; measure what the blanks hide.

What You Have Now

factory/adopt/comparison-board.md - a seven-dimension comparison of the factories you studied, plus the pattern you copy and the gap you measure yourself. You now hold a board with a source in every cell, a one-row choice of the pattern you will copy with a test that can fail, and a five-row gap table with numerators, denominators and owners. The validator keeps empty cells out of all three. Keep the file beside your scoreboard from lesson 14 and update it when a company publishes a number you marked "not published".

Further Reading

• Intercom, "AI is approving our pull requests": the only published no-human lane, with the numbers the board marks as uncontrolled.
• Anthropic, "Code Review": price and time per review, and the advisory-reviewer design.
• Stripe, Minions part 2: the capped CI loop and the weekly count.
• Cloudflare, "AI code review": risk tiers, specialists and the tracked override.
• Microsoft .NET, ten months with the coding agent: a merge rate with a denominator, on one repo.
• Habituation at the Gate, arXiv preprint: why a human gate needs sampling and rotation.

Exercises

1. Fill one row. Take the company page you read most closely and write its row of the board from memory of the page, then check each cell against the page. Mark every cell you could not source "not published".
2. Fill your own row. Write "Your line today" in the board. For each of the seven dimensions, cite where in your repo or tooling the fact lives, or write "not published" if it lives in nobody's head.
3. Run the gap table. From the last 30 days of git history and your provider bill, compute the five missing numbers where you can. Write the numerator and denominator, and write "not available" where you cannot. State which would make you pause a lane.

Check your understanding