Practice project: Project 02. Add a Verifier and Durable State

Lecture 07. The Most Valuable Node Produces Nothing

extract liability was asked to review its own output before passing it downstream — a quick self-check appended to the same prompt: "confirm the clause you extracted is accurate before returning it." It confirmed its own extraction was accurate in every single run, including the one where it had invented a liability cap that didn't appear anywhere in the contract. The self-check wasn't lying. It was drawing on the exact same reading of the contract that produced the wrong extraction in the first place — of course it agreed with itself.

That's the general failure, not a one-off: a review drawn from the same context that produced the work will catch formatting problems and miss being wrong. It has no independent evidence to disagree with.

What a Verifier Actually Adds

The highest-leverage node in a graph that survives production is a verifier — and it adds no content of its own. Its entire job is to stop weak work from moving downstream, and it does that by checking the work against something the generator didn't have access to when it produced the work.

For extract liability, that something is the source text itself:

interface VerifyResult {
  verdict: "confirmed" | "unsupported" | "contradicted";
  citedText: string | null;   // the exact source sentence the claim maps to
}

function verifyExtraction(
  extracted: ExtractedClause,
  sourceText: string,          // the verifier reads the ORIGINAL contract, not the extractor's summary
): VerifyResult {
  if (extracted.text === null) return { verdict: "confirmed", citedText: null };
  const found = sourceText.includes(extracted.text.trim());
  return found
    ? { verdict: "confirmed", citedText: extracted.text }
    : { verdict: "unsupported", citedText: null };   // extractor claimed text the source doesn't contain
}

Notice what verifyExtraction reads: sourceText, the original contract — never extracted's reasoning about why it pulled that text out. That's not an implementation detail. It's the entire point.

Fresh Context Is Not Optional

A verifier that shares a context window with its generator is not a verifier — it's the same reasoning, asked twice. The clause-citation check above only works because it never sees extract liability's prompt, its intermediate reasoning, or its confidence. It sees exactly two things: the claim, and the source it's supposed to be grounded in. That's a structurally separate node with its own contract (Lecture 03), not a second pass inside the same call.

This generalizes past clause extraction. A verifier can check:

• Whether every claim in a generated summary carries a citation, and whether the citation actually supports the claim.
• Whether an output matches its declared schema (Lecture 03's contract, checked mechanically, not just trusted).
• Whether a second, independently-run path reached the same conclusion as the first.
• Whether generated code's tests pass — a verifier that runs pytest, not one that reads the code and decides it looks right.

Every one of these shares the same shape: a check against something external to the generator — source text, a schema, a second independent run, an actual test execution — never a re-read of the generator's own reasoning.

Separate the Roles, Separate the Failure Boundaries

Don't ask one agent to produce, approve, and publish in the same context — it will approve. This is the same gate logic as merging code: something outside the worker decides whether the work is allowed to continue, and that something has to be a genuinely separate process, not a second instruction inside the same one.

Put the verifier on the edge — between the generator and everything that reads its output — not inside the generator:

check liability never sees an unverified extraction. The verifier's unsupported output routes to re-extract — a controlled cycle (Lecture 04) — before anything downstream trusts the claim.

Key Takeaways

• A verifier adds no content — its entire job is to stop weak work from moving downstream. That's what makes it the highest-leverage node in the graph, not the flashiest one.
• A review drawn from the same context as the work will catch formatting and miss being wrong. It has no independent evidence to disagree with the reasoning that produced the mistake.
• Verification means checking against something external: source text, a schema, a second independent run, or an actual test execution — never the generator's own reasoning trace.
• Put the verifier on the edge, in a fresh context, not inside the generator. One agent that produces, approves, and publishes in the same context will approve.

What You Have Now

A standalone verifier node, running in its own context, gating one generator's output in your graph. It sits on the edge leaving your riskiest generator node in the design document, not inside it. This is the center of the Verify stage: the check that catches wrong work before it moves downstream.

Save it as graph/lecture-07.md; the lab's checker looks for that file at that path.

Exercises

1. Find a self-check that shares context. In a workflow you run, find a place where the same call that produced an output also approves it. Redesign it as a separate node with a fresh context, checking against something external — source text, a schema, a test run.
2. Write one verifier's contract. Using Lecture 03's four-line format (job, input, output, failure states), write the contract for a verifier node in your own graph. Its input must include something the generator's context doesn't carry forward — name specifically what that is.

Check your understanding