Lecture 06. Let the Model Judge, Let the Graph Decide

The classify risk node from Lecture 04's router was given one instruction: "read the four clause verdicts and decide what happens next: file it, send it to deeper review, or escalate to a human." It worked, mostly. Then a contract with a slightly unusual but harmless termination clause got escalated to a human because the model, reasoning in the moment, decided "unusual" was close enough to "risky" to be safe. Another contract with a genuinely alarming liability cap got auto-filed because the model's phrasing that day leaned the other way. Same inputs, different runs, different verdicts, because the model wasn't just judging the contract. It was also deciding, on its own, what to do about what it found.

Those are two different jobs, and collapsing them into one node is where routing stopped being predictable.

Split the job in two

Classification is the job a model is good at: read ambiguous, unstructured evidence and produce a label. "Given these four clause verdicts, is this contract's risk low, medium, or high?" is exactly the kind of judgment a model should make: it's probabilistic, it benefits from the model's ability to weigh imperfect signals, and it's the same kind of task the model is already doing well inside check liability and check termination.

Routing is a different job: given a label, decide what happens next. That's not a judgment call, it's a lookup.

type RiskLabel = "low" | "medium" | "high";

// The model's only job: produce a label, nothing else.
function classifyRisk(verdicts: CheckResult[]): RiskLabel {
  return callModel("classify overall risk from these clause verdicts", verdicts);
}

// The route table's only job: map a label to a path. No model call, no judgment.
const ROUTE_TABLE: Record<RiskLabel, NextNode> = {
  low: "file_contract",
  medium: "deeper_review",
  high: "human_escalation",
};

function route(label: RiskLabel): NextNode {
  return ROUTE_TABLE[label];   // deterministic — same label, same path, every time
}

Two functions instead of one. classifyRisk is probabilistic and can be wrong the way any model judgment can be wrong. route cannot: the same label produces the same path every single time, because it's a table lookup, not a decision.

Only the diamond is a model call. Every arrow leaving it is a table lookup, the same label taking the same path every time.

What this buys you

Predictability. Given the same label, the route is always the same. If a contract keeps getting sent to the wrong path, the bug is isolated immediately: either classifyRisk produced the wrong label (a model problem, fix the classifier or its inputs), or ROUTE_TABLE maps the label to the wrong path (a table problem, fix one line). Before the split, both possibilities were tangled inside one model call and neither was easy to isolate.

Explainability. When a contract ends up on the wrong path, you can point at a specific label and a specific table row, instead of re-reading a paragraph of reasoning that the model produced once and can't reliably reproduce. "It was classified medium, and medium routes to deeper review" is an answer someone can audit in five seconds. "The model reasoned about it and decided" is not.

Cheap changes. The business decides tomorrow that medium should escalate to a human instead of going to deeper review. That's one line in ROUTE_TABLE, no re-prompting, no re-testing the classifier's judgment, because the classifier's job never changed.

Where the line actually falls

The split isn't "models are unreliable, so remove them from decisions." It's about which decision each part makes. A model classifying "is this evidence closer to low, medium, or high risk" is doing something models are legitimately good at: weighing partial, ambiguous signals. A model deciding "and therefore the contract should go to path X" is doing something a table does more reliably, more cheaply, and more auditably, once the label already exists.

Anything unrecognized by the table deserves its own row, not a silent fallback: RiskLabel should be an exhaustive union, and ROUTE_TABLE should cover every member of it. If the classifier ever returns something outside "low" | "medium" | "high", that's a contract violation (Lecture 03): treat it as a named failure, not a case the router improvises around.

Faded case

Skip to the open case if this is familiar. A support-ticket triage node reads a ticket's text and, in one model call, both labels its severity as low, medium, or high and decides what happens next: auto-reply, queue for a human, or page on-call. Two tickets arrive minutes apart with nearly identical wording — same product, same error message, same customer tier — and come out of the same node with different verdicts: one gets auto-replied, the other gets queued for a human. Nothing about either ticket's actual severity changed between the two calls; only the model's phrasing that run did, and the team has no way to point at a specific rule and say which ticket was wrong. Decide now: what would you split this node into, and which half gets a table instead of a model call?

Open case

Find a node in your own graph that both judges evidence and decides what to do about it in the same call — Exercise 1 asks for exactly that. Split it the way this lecture and the faded case above both did, then check whether the table you write out actually covers every label the classifier can return.

Key takeaways

• Classification and routing are different jobs. A model judging ambiguous evidence into a label is the right use of a model; deciding what a label means for the workflow is a lookup, not a judgment.
• The route table is the single source of truth for label → path. If two runs with the same label take different paths, something outside the table is re-deciding: find it and remove it.
• The split makes bugs isolatable. Wrong path taken means either the classifier mislabeled (fix the model call) or the table maps wrong (fix one line), never both tangled together.
• Policy changes become table edits. Changing what a label routes to doesn't touch the classifier at all.

What You Have Now

A route table that separates your classifier (model, probabilistic) from your router (table, deterministic). It replaces whatever single node in your design document used to do both jobs, and it's what makes the Verify stage's checks auditable — a wrong path traces to one label or one table row, never both at once.

Save it as graph/lecture-06.md; the lab's checker looks for that file at that path.

Exercises

1. Find a node doing both jobs. In a workflow you run, find a node that both judges evidence and decides what to do about the judgment in the same model call. Split it into a classify function and a route table, the way this lecture did.
2. Make the route table exhaustive. For your classifier's label type, write a route table that covers every possible label, including ones you consider unlikely. Anything the classifier could return that the table doesn't cover is a gap this lecture calls a contract violation, not an edge case to improvise around later.

Check your understanding