Lesson 02. Most Tasks Do Not Deserve a Loop

A team spent a full week wiring an overnight loop to draft a friendly morning digest of support tickets for their founder: read the tickets, summarize the mood, flag anything scary, email it at 7 a.m. It ran every night for two months. Nobody could say whether any single night's digest was better or worse than the one before; "good" meant "the founder didn't complain," and the founder read maybe one in five. The loop burned a modest but real amount of model spend every night, improved nothing measurably, and when the founder finally asked for a change, shorter, bullet points instead of prose, there was no loop-score the change could move, so the team rewrote the prompt by hand, exactly like they would have with no loop at all. Two months of upkeep had gotten them to where a single saved prompt template would have gotten them in an afternoon. Decide now: what, before writing a single line of automation, would have told this team the task wasn't shaped like a loop in the first place?

This lesson gives you a loop-worthiness checklist card: four yes/no questions you run against any candidate task before you spend an hour building automation for it. It's the cheap screen that comes before every other lesson in this module, the check you run before deciding whether a locked scorer, a ratchet ledger, or a tiered budget is even worth building for this particular task.

What is actually going on

Not every recurring annoyance is a loop. A loop is worth building when four separate things are true about the task, and it stops being worth building the moment any one of them isn't, no matter how tempting the rest look.

Repeated. The same shape of task has to recur, without you rewriting the brief each time. A one-off redesign is not a loop candidate, no matter how many hours it takes; it's a project.

Token budget named. Someone has put a number on it: a token cap, a dollar figure, a wall-clock limit per run. "Whatever it takes" is not a budget, it's an invitation for the loop to spend without a ceiling, and a loop without a named budget can never be compared against the manual version it's replacing.

Machine-checkable score. Something other than a person's gut has to say whether a given run's output was good: a test suite, a labeled set of past correct answers, a rule you can write down and check against. If the only available judge is "does this feel right to whoever reads it," you don't have a score, you have an opinion that happens to recur.

Agent can run its own output. The agent, or a verifier right next to it, has to be able to execute, or at least mechanically check, the thing it produced, not just describe it in prose. An output only a human can evaluate by reading it carefully is an output a loop can generate quickly but never actually verify.

All four together turn "a thing I do by hand a lot" into "a thing worth automating into a loop." A loop without a machine-checkable score doesn't run faster, it just fails more confidently, and that single missing gate is enough to sink an otherwise well-built loop.

The diagram below treats the four as sequential gates, because that's how to apply them in practice: check the cheapest, most disqualifying one first, and stop the moment one fails.

diagramsource

drawing

Four gates, checked in order, is the whole mechanism. Everything else in this lesson is applying it to real-looking candidates and noticing which gate actually does the killing.

The rule and its condition

Build the loop when all four gates clear. File the task as "watch," name the one gate that failed, and revisit it only when that specific gate can be checked again. Don't build a weaker loop around a task that fails one gate, hoping the other three carry it: a loop missing even one of the four doesn't run slower or worse, it runs exactly as fast and confidently as a fully loop-worthy one, with no way to tell whether any given night's output was actually an improvement.

The gate that failed matters most in practice, because the fix is different for each:

Failed gate	What "watch" means	What unblocks it
Repeated	Shelve it, run it once well	It starts recurring on its own, unprompted
Token budget named	Don't build yet	Someone writes down a number
Machine-checkable score	Don't build yet	A labeled dataset or rule gets built first
Agent can run its own output	Don't build yet, or redesign the output	Output changes to something executable or checkable

Three of the four failures are fixed by a small piece of work before the loop, not by building it anyway and hoping the missing piece shows up. Only the first, not actually repeated, has no fix; it simply means this was never loop material.

Worked case

Situation. A small ops team has three tasks on their board they're considering turning into overnight loops, and one afternoon to decide which, if any, are worth the setup cost.

1. Redesign the onboarding flow for next quarter, based on the last round of user interviews.
2. Every morning, read overnight support tickets and draft a friendly summary email for the founder.
3. Every time a refund-request ticket comes in, check it against the written refund policy, decide approve or deny with the specific policy clause cited, and file the refund in the billing system if approved.

Decide now, before reading the table: which one, if any, actually clears all four gates?

Only the third does. Here is the table they built to check it, which doubles as this lesson's deliverable, a loop-worthiness checklist card, filled in three times:

Gate	1. Onboarding redesign	2. Morning digest email	3. Refund triage
Repeated, same shape	No, one-time project	Yes, daily	Yes, per ticket
Token budget named	n/a, gate 1 already failed	No, "whatever reads well"	Yes, capped per ticket
Machine-checkable score	n/a	No, "founder's gut"	Yes, decision checked against written policy clauses and a labeled set of past resolved tickets
Agent can run its own output	n/a	No, only a human reading the email judges it	Yes, it files the refund; the billing system's accept or reject response is itself a pass/fail signal
Verdict	Watch: not repeated	Watch: no budget, no score	Build the loop

Candidate 1 dies at the first gate, and it costs nothing further to evaluate once gate 1 fails, which is the whole point of checking gates in order instead of scoring every candidate up front.

Candidate 2 is the trap: it feels loop-shaped because it recurs every day, and recurrence is the gate people check first and stop at. It fails the two gates that actually determine whether the loop would be worth running: nobody put a number on what it should cost, and nothing except a founder's own attention, itself unmeasured, says whether any given morning's draft was good. This is the exact task from this lesson's opening failure.

Candidate 3 clears all four, and the reasoning is specific, not just "it has rules": refund decisions recur constantly, the team can name a real per-ticket budget because the decision is bounded, a machine-checkable score exists because the decision is checkable against the policy's own stated clauses and against a backlog of already-resolved tickets with known correct outcomes, and the output is not prose someone has to read and vibe-check, it is an action with its own pass/fail signal from the billing system's response.

Illustrative composite. From a real audit of a self-improving dev crew's adjacent intake process, 2026-10-04: of roughly 20 candidate automations proposed over one quarter, only 3 cleared all four gates on first pass; the most common single failure, in 11 of the remaining 17, was a missing machine-checkable score, not a missing budget or a lack of repetition.

Here is the shape of the loop candidate 3 becomes once it clears the screen, the order each ticket actually moves through:

diagramsource

drawing

Every stage in that diagram is either a lookup against a written document or a check against a system's own response; nothing in it requires a person to read prose and decide whether it felt right. That's what "agent can run its own output" actually buys: a loop that fails loudly and specifically, instead of one that produces something each morning and hopes.

Faded case

Skip to the open case if this is familiar. A different team is considering a fourth candidate: every week, scan the dependency manifest, bump anything with a newer patch version, run the full test suite, and open a PR only if everything passes, otherwise leave a note explaining what broke. It recurs weekly, the team is willing to name a budget, one test-suite run's worth of compute, capped, and the agent clearly executes its own output, since it runs the test suite itself and pass or fail is immediate. Walk it through all four gates yourself before reading the check below, and decide whether this one builds or watches, and if it watches, name the gate.

Open case

Pick one task on your own team's board that keeps getting pitched as "we should automate this." Run it through the four gates in order, stop at the first one that fails, and write down which gate that was. If all four clear, don't build the loop yet either; just write down the budget number, the score definition, and the one sentence describing how the agent will check its own output, before you write a single line of automation.

Where this breaks

A narrow, easily machine-checkable score can pass the third gate while measuring something shallow. Clearing the four-test proves the task is loop-shaped, not that the score you picked is a good one, which is exactly the problem this module's locked-scorer lesson protects against once the loop is actually running.

The gates are a screen, not a guarantee of value: a task can clear all four and still not be worth the engineering time relative to how often it actually comes up. Loop-worthiness is necessary, not sufficient.

Teams under pressure to "automate something" tend to grade their own candidate generously on the two subjective-looking gates, budget and score. Running the screen past a second person catches the self-serving "yes" a solo pass tends to produce.

Key Takeaways

• Run the four gates in order and stop at the first failure; don't average them into a vibe.
• Recurrence is necessary but routinely mistaken for sufficient; check the other three before building anything.
• A loop without a machine-checkable score doesn't run faster, it just fails more confidently.
• Name the token budget and the scoring rule before the loop, not after you see what it produces.
• Clearing the four-test proves a task is loop-shaped, not that the loop is worth the time or that its score is a good one.
• File anything that fails a gate as "watch," named by which gate, not as a flat "no."

What You Have Now

You have a loop-worthiness checklist card: four yes/no gates, repeated, token budget named, machine-checkable score, agent can run its own output, checked in order, with the specific failing gate recorded instead of a flat no. Keep it next to your team's backlog or idea board; it's the cheap screen every candidate automation runs through before anyone spends an hour on a locked scorer, a ratchet ledger, or a tiered budget for it. This is the triage stage of the module: everything after this lesson assumes the task in front of it already cleared this card.

Further Reading

• Hyperband — the successive-halving pattern behind checking the cheapest, most disqualifying gate first and stopping at the first failure, instead of spending full evaluation effort on every candidate.
• AlphaEvolve — evaluation cascades that run cheap case tiers before expensive ones, the same cheap-screen-before-spend logic this lesson's four gates apply to candidate tasks.
• GEPA — keeps a Pareto frontier instead of collapsing to one aggregate score, a direct counter to the "Where this breaks" risk of a narrow score that passes the gate while measuring something shallow.
• Goodhart Taxonomy, Manheim & Garrabrant — names the specific ways a machine-checkable score can still be a bad score, useful once a task clears gate three and you're deciding how much to trust what it checks.

Exercises

1. Take three tasks currently on your backlog pitched as "automate this" and run each through the four-gate checklist card, in order, recording which gate, if any, kills each one.
2. For one task that clears all four gates, write the actual budget number, the actual scoring rule, and the one sentence describing how the agent checks its own output, before building anything.
3. For one task that fails on "machine-checkable score" only, spend thirty minutes building the smallest possible labeled dataset or rule that would let it pass, then re-run the gate.

Check your understanding