Practice project: Project 04. Use runtime feedback to correct agent behavior

Lecture 07. Draw Clear Task Boundaries for Agents

You tell Claude Code to "add user authentication to this project," and it starts modifying the database schema, writing routes, changing frontend components, and, while it's at it, refactoring the error-handling middleware. Two hours later you check: 12 files modified, 800 lines of new code, and not a single feature works end-to-end.

Agents are born with an impulse to "do a little extra", they see related things and just handle them along the way. The problem is, doing too many things at once almost guarantees none of them get done well.

Anthropic's "Effective harnesses for long-running agents" engineering blog states clearly: when prompts are too broad, agents tend to "start multiple things at once" rather than "finish one thing first." OpenAI's Codex engineering practices found the same, tasks without explicit scope controls see completion rates plummet. This is not a model problem, it's a harness problem. You didn't draw the boundary.

Attention Is a Finite Resource

This isn't a metaphor, it's math. Assume the agent's context capacity is C and it activates k tasks simultaneously. Each task gets an average of C/k reasoning resources. When C/k drops below the minimum threshold needed to complete a single task, none of them get finished.

Claude Code's real behavior is telling. Ask it to "add user registration" and it might:

1. Create a User model
2. Write the registration route
3. Notice it needs email verification, so add a mail service
4. See that passwords need hashing, so bring in bcrypt
5. Notice the error handling is inconsistent, so refactor the global error middleware
6. See the test file structure is messy, so reorganize the directory

Six steps later, every one is half-done. No end-to-end verification, complex coupling between the half-baked code, and the next session to pick up the pieces will be completely lost.

Anthropic's harness guidance points the same direction: prompt for the smallest next step rather than the broad goal, because agents given broad prompts reliably start more than they finish. The tell is visible in any transcript log: sessions that produce the most code are rarely the sessions that complete the most features, volume and completion are different axes, and overreach trades the second for the first.

The Rule and Its Condition

Set the WIP limit from the C/k math above, not from a fixed number copied out of Kanban. Use WIP=1 when the largest task in your queue needs more than roughly a third of one session's context budget to reach a verified state, true for nearly every real feature. Raise the limit only for a queue of tasks small enough that C/k still clears that same finish threshold at k tasks, and confirm it against a real solo run before trusting it, because the threshold moves with the work, not with the number you'd prefer.

The condition is observable in your own transcripts: run the task alone once, see how much of a session's budget it actually consumed to reach passing, then compare that against C/k for the k you're considering activating together.

WIP=1 Workflow

Core Concepts

• Overreach: The agent activates more tasks in a single session than optimal. This isn't subjective, it's quantifiable: doing 5 features with 0 passing end-to-end is overreach.
• Under-finish: The ratio of tasks that pass end-to-end verification, out of all activated tasks, falls below threshold. Code written but tests not passing is under-finish.
• WIP Limit (Work-in-Progress Limit): From Kanban methodology. Core idea: limit how many tasks are in-flight at once. For agents, WIP=1 is the safest default, finish one before starting the next.
• Completion Evidence: The verifiable condition a task must satisfy to move from "in progress" to "done." Without this, agents substitute "the code looks fine" for "the behavior passes tests."
• Scope Surface: A DAG structure where each node is a work unit and edges are dependencies. States are limited to four: not_started, active, blocked, passing.
• Completion Pressure: The constraining force the harness exerts through WIP limits and completion evidence requirements, forcing the agent to finish the current task before starting a new one.

Overreach and Under-finish Are Two Sides of the Same Coin

These two problems aren't independent, they amplify each other. Overreach dilutes attention, diluted attention causes under-finish, and the half-finished code left behind increases system complexity, which further drives overreach in the next task. A vicious cycle.

In Kanban terms: Little's Law tells us L = lambda * W. If work-in-progress L is too high (doing too many things at once), the lead time W for each task inevitably increases. For agents, this means each feature takes longer from start to verified completion, and the probability of failure grows.

This is an old problem in the human world too. Steve McConnell documented in Rapid Development that scope creep is the leading cause of project failure. But humans at least have the intuition of "I've done enough." Agents have none. Generating the next idea costs the model almost nothing, writing "let me fix this too while I'm here" barely costs extra tokens, but every additional modification dilutes the agent's attention.

How to Do It Right

1. Enforce WIP=1

This is the most direct and effective method. In your harness, tell the agent explicitly: only one task is allowed in "active" status at any time. In Claude Code's CLAUDE.md or Codex's AGENTS.md, write:

2. Define Explicit Completion Evidence for Every Task

Done is not "code is written", it's "behavior verification passes." In your feature list, every entry needs a verification command:

3. Externalize the Scope Surface

Use a machine-readable file (JSON or Markdown) to record all task states. Any new session can read this file and immediately know: which task is active? What behavior counts as done? What verifications have passed?

4. Monitor Verified Completion Rate

The harness should continuously track VCR (Verified Completion Rate) = verified tasks / activated tasks. Block new task activations when VCR < 1.0.

Trace It: 5 Features at Once vs. WIP=1

A REST API project, 8 features. Session 1: the agent activates 5 features simultaneously, no WIP limit.

Predict before reading on: how much of that session's output passes end-to-end tests?

Illustrative composite. Trace it against VCR from above: activating 5 at once means 5 unfinished surfaces competing for the same context and the same verification budget in one pass, nothing forces any single one to reach "curl returns 201" before the next starts. Actual result: ~800 lines across 12 files, but only user registration's end-to-end tests pass, the rest have schemas without validation, routes returning the wrong response shape. By session 3, only 3 of 8 features are complete.

Now cap WIP at 1: session 1 does user registration only, ~200 lines, 100% of its end-to-end tests pass, commits a clean verified implementation. By session 4, 7 of 8 are complete.

Faded case

Skip to the open case if this is familiar. Same team, second sprint: 6 features left, and this time they enforce WIP=1 from the start. Session 1 activates password reset only, ships a verified end-to-end pass, commits clean. Session 2 activates email verification, also passes, also commits clean. Going into session 3, a stakeholder asks for two quick features to demo Friday, notification preferences and account deletion, both small. The agent proposes activating both at once, since each one alone looks like "maybe an hour's work," and the WIP=1 rule only ever said "one task in active status," not "one task in a session."

Decide: does the agent's proposal actually satisfy WIP=1, and what should the team do about the Friday deadline instead?

Open case

Look at whatever your agent is doing in your own repo right now. How many features or tasks does it currently show as "active" or "in progress" at once? If the answer is more than one, that's the overreach this lecture describes, not a special case: pick the one closest to a real end-to-end pass, park the rest, and let the WIP=1 discipline the trace and the faded case above walked through apply to your own backlog. Exercise 1 below turns the same check into a full atomization exercise if you want more structure.

Key Takeaways

• WIP=1 is the default safe setting for agent harnesses, finish one, then start the next; don't try to parallelize.
• Completion evidence must be executable, "the code looks fine" doesn't count; "curl returns 201" does.
• The scope surface must be externalized as a file, not just mentioned in conversation, but recorded in a machine-readable format in the repo.
• Overreach and under-finish are symbiotic, solving one solves the other.
• "Do less but finish" always beats "do more but leave half-done", agent code lines and feature completion rate are negatively correlated. Quality always beats quantity.

Example Files

Copy-ready course examples referenced in this lecture.

next-task-template.md

A four-line task brief, feature, why it's next, what counts as passing, and (the load-bearing line) what must NOT be changed during this step.

# Next Task Template

- Current highest-priority feature:
- Why this feature is next:
- What counts as passing:
- What must not be changed during this step:

scope-surface-example.md

The same task written as a bad scope ("implement indexing") versus a good one, five bounded work units an agent can activate one at a time.

# Scope Surface Example

Task:

- Add indexing to the Electron knowledge app

Bad scope shape:

- “Implement indexing”

Better scope shape:

- Parse imported documents
- Split documents into chunks
- Persist chunk metadata
- Expose indexing status in the UI
- Add a reindex action

scope-tracker.ts

Runnable drift detector: given a feature list with one active feature and a ten-step change log, it flags every change claiming a non-active feature (four DRIFT entries in the sample), the mechanical version of "every changed line traces to the active task".

/**
 * scope-tracker.ts
 *
 * Reads a feature list and a change log. Enforces single-active-feature
 * policy. Given a log of changes, flags any changes outside the active
 * feature scope. Demonstrates how scope drift happens and how the tracker
 * catches it.
 *
 * Run: npx tsx docs/lectures/lecture-07-why-agents-overreach-and-under-finish/code/scope-tracker.ts
 */

// ---------------------------------------------------------------------------
// Types
// ---------------------------------------------------------------------------

interface Feature {
  id: string;
  name: string;
  status: "active" | "pending" | "done";
}

interface ChangeLogEntry {
  step: number;
  file: string;
  description: string;
  featureId: string; // The feature this change claims to belong to
}

// ---------------------------------------------------------------------------
// Sample data
// ---------------------------------------------------------------------------

const features: Feature[] = [
  { id: "F-001", name: "Search endpoint", status: "active" },
  { id: "F-002", name: "Delete endpoint", status: "pending" },
  { id: "F-003", name: "Rate limiting", status: "pending" },
  { id: "F-004", name: "User dashboard", status: "pending" },
];

// A realistic change log where the agent gradually drifts from the active feature
const changeLog: ChangeLogEntry[] = [
  { step: 1, file: "src/routes/search.ts", description: "Add search route handler", featureId: "F-001" },
  { step: 2, file: "src/routes/search.ts", description: "Add query parameter validation", featureId: "F-001" },
  { step: 3, file: "src/routes/search.ts", description: "Add search results pagination", featureId: "F-001" },
  { step: 4, file: "src/routes/delete.ts", description: "Add delete route handler", featureId: "F-002" }, // DRIFT
  { step: 5, file: "src/middleware/rate-limit.ts", description: "Add rate limiter middleware", featureId: "F-003" }, // DRIFT
  { step: 6, file: "src/routes/search.ts", description: "Integrate rate limiter into search", featureId: "F-001" },
  { step: 7, file: "src/dashboard/ui.tsx", description: "Create dashboard layout component", featureId: "F-004" }, // DRIFT
  { step: 8, file: "src/routes/search.ts", description: "Add search response formatting", featureId: "F-001" },
  { step: 9, file: "src/routes/delete.ts", description: "Add delete confirmation logic", featureId: "F-002" }, // DRIFT
  { step: 10, file: "src/routes/search.ts", description: "Add search tests", featureId: "F-001" },
];

// ---------------------------------------------------------------------------
// Scope tracker
// ---------------------------------------------------------------------------

interface ScopeCheckResult {
  step: number;
  file: string;
  description: string;
  featureId: string;
  inScope: boolean;
  activeFeature: string;
}

function trackScope(
  featureList: Feature[],
  changes: ChangeLogEntry[]
): ScopeCheckResult[] {
  const activeFeatures = featureList.filter((f) => f.status === "active");
  const activeIds = new Set(activeFeatures.map((f) => f.id));
  const activeNames = activeFeatures.map((f) => f.name).join(", ");

  return changes.map((change) => ({
    step: change.step,
    file: change.file,
    description: change.description,
    featureId: change.featureId,
    inScope: activeIds.has(change.featureId),
    activeFeature: activeNames,
  }));
}

// ---------------------------------------------------------------------------
// Reporting
// ---------------------------------------------------------------------------

function pad(s: string, len: number): string {
  return s.length >= len ? s : s + " ".repeat(len - s.length);
}

function run(): void {
  const results = trackScope(features, changeLog);

  console.log("\n" + "=".repeat(100));
  console.log("  SCOPE TRACKER -- Single Active Feature Enforcement");
  console.log("=".repeat(100));

  console.log("\n  Active feature: " + features.filter((f) => f.status === "active").map((f) => `${f.id} (${f.name})`).join(", "));
  console.log("  Pending features: " + features.filter((f) => f.status === "pending").map((f) => `${f.id} (${f.name})`).join(", "));

  // Detailed change log
  console.log("\n" + "-".repeat(100));
  const header = `| ${pad("Step", 5)}| ${pad("File", 35)}| ${pad("Description", 40)}| ${pad("Feature", 8)}| ${pad("In Scope", 10)}|`;
  const sep = `|${"-".repeat(7)}|${"-".repeat(37)}|${"-".repeat(42)}|${"-".repeat(10)}|${"-".repeat(12)}|`;
  console.log(header);
  console.log(sep);

  let inScopeCount = 0;
  let driftCount = 0;

  for (const r of results) {
    const scopeLabel = r.inScope ? "OK" : "DRIFT";
    if (r.inScope) inScopeCount++;
    else driftCount++;

    const marker = r.inScope ? "  " : ">>";
    console.log(`${marker}| ${pad(String(r.step), 5)}| ${pad(r.file, 35)}| ${pad(r.description, 40)}| ${pad(r.featureId, 8)}| ${pad(scopeLabel, 10)}|`);
  }
// … trimmed for the lecture — the pattern continues in the same shape

What You Have Now

A next-task file with an explicit scope surface and exclusions.

The exclusions are the point: a task file that only says what to do still leaves the agent free to "improve" everything nearby. Name what is out of bounds, and the diff you get back starts matching the diff you asked for.

Save it as harness/next-task.md; the lab's checker looks for that file at that path.

Further Reading

• Effective harnesses for long-running agents - Anthropic, Anthropic's engineering blog, detailed discussion of the "small next step" strategy
• Harness Engineering - OpenAI, OpenAI's complete treatment of harness engineering
• Kanban: Successful Evolutionary Change - David Anderson, The classic source on WIP limits
• Rapid Development - Steve McConnell, Empirical data on scope creep as the leading cause of project failure

Exercises

1. Task Atomization: Pick a broad requirement (e.g., "implement a user management system") and break it into at least 5 atomic work units. For each unit, specify: (a) a single behavior description, (b) an executable verification command, (c) dependencies. Check whether the decomposition satisfies the WIP=1 constraint.

2. Comparison Experiment: Run the same project twice, once without constraints, once with enforced WIP=1. Compare verified completion rate, total lines of code, and effective code ratio.

3. Completion Evidence Audit: Review a recent agent run's output, classifying each code change as "completed behavior," "incomplete behavior," or "scaffolding." Add missing verification commands for each incomplete behavior.

Check your understanding