Practice project: Project 06. Build a Complete Agent Workspace

Lecture 12. Leave a Clean Handoff at the End of Every Session

Your agent runs all afternoon, modifies 20 files, commits the code, and the session ends. The next agent session starts up and immediately discovers: build is broken, tests are red, temporary debug files are scattered everywhere, the feature list hasn't been updated, and progress is completely opaque. The first 30 minutes of the new session are spent entirely on "figuring out what the last session actually did."

Both OpenAI and Anthropic state clearly: long-term reliability depends on operational discipline, not just single-run success. The quality of state at the end of each session directly determines the next session's efficiency.

Entropy Growth Is the Default State

Lehman's laws of software evolution tell us that systems undergoing continuous change will inevitably grow more complex unless actively managed. This is especially true for AI coding agents. Every session introduces changes, and without cleanup at exit, technical debt accumulates exponentially.

During five months of Codex experiments, OpenAI observed something striking: agents copy patterns already present in the repository, even when those patterns are inconsistent or suboptimal. Over time, this copying inevitably leads to drift. The first person leaves a coffee cup in the common area; the second person figures "it's already messy" and leaves another; a week later the table is buried under cups. A codebase works the same way.

The OpenAI team initially spent every Friday — 20% of the work week — manually cleaning up "AI slop" (OpenAI: Harness engineering: leveraging Codex in an agent-first world, observed 2026-09-13), but this approach clearly doesn't scale. They eventually arrived at a systematic solution:

1. Encode "golden rules" into the repository: Rules like "prefer the shared utility package over hand-rolled ad-hoc helpers" (keep invariants centralized) and "don't YOLO-guess data structures" (validate boundaries or depend on typed SDKs). These rules are concrete, mechanical, and automatically checkable.
2. Establish periodic cleanup workflows: A fleet of background Codex tasks that regularly scan for deviations, update quality scores, and open targeted refactoring PRs. Most can be reviewed and auto-merged within a minute.
3. Capture human taste once, enforce it continuously: Review comments, refactoring PRs, and user-facing bugs are all translated into documentation updates or encoded directly into tooling. When documentation isn't enough, promote the rule into code.

Technical debt is a high-interest loan. Continuously paying it off in small increments is almost always better than letting it accumulate into one massive payoff event.

Source: OpenAI: Harness engineering: leveraging Codex in an agent-first world

Clean State: More Than "The Code Compiles"

Clean state isn't simply "the code compiles." Building without errors is the most basic requirement; the next session shouldn't have to fix build errors first. All tests must pass too, including tests that existed before the session; the session is responsible for not breaking existing functionality. And this should be verified in CI, not just "works on my machine."

But that's still not enough. Current progress must be recorded in machine-readable artifacts: completed subtasks with their passing criteria, in-progress but incomplete subtasks with current state, and not-yet-started subtasks. Good progress records collapse session-startup diagnosis from an excavation into a read. Temporary artifacts (debug logs, temporary files, commented-out code, TODO markers) must also be cleaned up, because they increase cognitive load for the next session. The standard startup path must remain functional too. Can the next session start working without manual intervention? Environment initialization, codebase loading, context acquisition, task selection: none of these paths can be broken.

Core Concepts

• Clean state: The system must satisfy five conditions at session end: build passes, tests pass, progress recorded, no stale artifacts, startup path available. Missing any one means the session isn't "done."
• Session integrity: Analogous to database transactions: either fully commit and leave a clean state, or roll back to the last consistent state. No middle ground.
• Quality document: An active artifact that continuously records quality ratings for each module. Not a one-time assessment, but a tracker showing whether the codebase is getting stronger or weaker over time.
• Cleanup loop: A regular maintenance session aimed at systematically reducing entropy in the codebase. Not an emergency fix, but routine operations.
• Harness simplification: As model capabilities improve, periodically remove harness components that are no longer necessary. A constraint essential today may be unnecessary overhead in three months.
• Idempotent cleanup: Cleanup operations produce the same result regardless of how many times they run, ensuring cleanup remains safe even in failure-retry scenarios.

"Clean Up Later" Means Never Clean Up

The most common mental trap is "no time to clean up this session, I'll do it next time." But the next agent session doesn't know what you left behind. It sees a mess of code and uncertain state. It'll spend significant time inferring "which parts of this code are intentional and which are temporary."

Worse, every session has its own task objectives. The new session is there to do new work, not clean up the previous session's mess. It'll ignore the chaos and start new work on top of it, introducing even more chaos. This is entropy's positive feedback loop.

A modeled 12-week trajectory makes the entropy curve concrete — the numbers show the shape of decay, not a measured study:

Illustrative composite — a modeled 12-week trajectory, not a measured study.

Without a cleanup strategy:

• Week 1: Build pass rate 100%, test pass rate 100%, new session startup 5 min
• Week 4: Build 95%, tests 92%, startup 15 min
• Week 8: Build 82%, tests 78%, startup 35 min
• Week 12: Build 68%, tests 61%, startup 60+ min

Same project with a cleanup strategy:

• Week 1: 100%, 100%, 5 min
• Week 12: 97%, 95%, 9 min

The point of the model is the divergence pattern: without cleanup, degradation compounds week over week; with cleanup, quality stays roughly flat for a small per-session cost. Teams that instrument these metrics (lecture 11) can plot their own curve, and it is worth doing, because the decay is invisible until you measure it.

How to Do It

1. Clean State Is a Necessary Condition for Completion

Define explicitly in the harness: session completion = task passes verification AND clean state check passes. Missing either one means the session isn't complete. Write in CLAUDE.md:

2. Dual-Mode Cleanup Strategy

Combine two cleanup modes:

Immediate cleanup (at end of every session): Clean up temporary artifacts created during the session, update feature list state, ensure build and tests pass. This is "reference counting" cleanup: clean up as soon as you're done using something.

Periodic cleanup (weekly): Full-system scan — handle accumulated structural issues, update quality documents, run benchmark tests to detect drift. This is "tracing" cleanup: a comprehensive maintenance pass done on a regular cadence.

3. Maintain a Quality Document

A quality document is an active artifact that continuously scores each module:

# Quality Document

## User Authentication Module (Quality: A)

- Verification passing: Yes
- Agent understandable: Yes
- Test stability: Stable
- Architecture boundaries: Compliant
- Code conventions: Followed

## Payment Module (Quality: C)

- Verification passing: Partial (payment callback untested)
- Agent understandable: Difficult (logic spread across 3 files)
- Test stability: Unstable (2 flaky tests)
- Architecture boundaries: Violations present
- Code conventions: Partially followed

New sessions read this document and immediately know where to prioritize. Fix the lowest-scoring module first.

4. Periodically Simplify the Harness

Every harness component exists because the model couldn't reliably do something on its own. But as models improve, these assumptions become outdated.

Anthropic's experiments demonstrated this directly. Their initial harness included a sprint-splitting mechanism — breaking work into small chunks for Sonnet 4.5 to complete one at a time. When Opus 4.6 shipped, the model's native capabilities could handle work decomposition autonomously, making sprint construction unnecessary overhead. After removing it, the builder agent could work continuously for over two hours without drifting, and the process felt smoother.

Source: Anthropic: Harness Design for Long-Running Application Development (2026-03-24)

But the evaluator told a different story. Even with Opus 4.6's stronger capabilities, when tasks approached the model's capability boundary, the evaluator still provided real value, catching the generator's missing functionality and stub implementations. This means the evaluator isn't a fixed yes/no decision; it depends on where task difficulty sits relative to model capability.

Recommended practice: Every month, pick one harness component, temporarily disable it, and run benchmark tasks. If results don't degrade, remove it permanently. If they do, restore it or replace it with a lighter alternative.

A deeper principle: as models improve, the interesting combinations in a harness shift rather than shrink. Problems that previously required explicit solutions get absorbed by model capabilities, but new capability boundaries open up harness design spaces that were previously impossible. The AI engineer's job is to continuously find the next valuable combination.

5. Cleanup Operations Must Be Idempotent

Cleanup scripts should be safe to run repeatedly, running them one more time shouldn't produce side effects:

# Idempotent cleanup operations
rm -f /tmp/debug-*.log  # -f ensures no error when files don't exist
git checkout -- .env.local  # Restore to known state
npm run test  # Verify cleanup didn't break anything

6. High Throughput Changes the Merge Philosophy

When agent output far exceeds human review capacity, the traditional merge philosophy needs adjustment. The OpenAI team's experience: in an environment where an agent opens 3.5 PRs per day (and later even more), minimizing blocking merge gates is the right call. PRs should be short-lived; test flakiness is usually resolved with subsequent runs rather than indefinitely blocking progress. In a system where the cost of fixing is low and the cost of waiting is high, moving fast with fast fixes is a better strategy than slow confirmation.

Caveat: This is irresponsible in a low-throughput environment. But when agent output far exceeds human attention, it's often the correct tradeoff. The key criterion: average cost of fixing a bug vs. average cost of waiting for a human to review a PR. When the former is lower than the latter, fast merging is the right call.

The Worked Comparison, End to End

Illustrative composite — the same modeled 12-week trajectory from earlier in this lecture, completed with both arms. Not a measured study; run the measurement on your own project.

An Electron app developed with agents over 12 weeks, comparing two approaches:

Without a cleanup strategy: quality decays on every axis: build and test pass rates sliding, session startup stretching from minutes to an hour, stale artifacts accumulating by the dozens.

With a cleanup strategy (full clean-state check at every session end, plus a weekly cleanup loop): pass rates hold near their week-1 levels, startup stays fast, and stale artifacts stay countable on two hands.

The asymmetry is the lesson: a few minutes of cleanup per session buys back hours of chaos later, and the cost stays constant while the savings compound.

Faded case

Skip to the open case if this is familiar. A different team's session just ended: the build passes, every test is green, and the feature list is updated. Nothing in the diff looks temporary. The agent didn't touch the dev-server startup path this session, and nobody has run the standard start command since two sessions ago. Decide now: does this count as a clean handoff under all five dimensions, and if not, name the one check the team is skipping.

Open case

Look at your own project's most recent agent session, whether it ended ten minutes ago or last week. Walk it against the five-dimension checklist above without adding anything new: build, tests, progress record, stale artifacts, startup path. Note which dimension you can't answer from what's already on disk. Exercise 1 below turns this same walk into a checklist you run for real, across five consecutive sessions.

Key Takeaways

• Clean state is a necessary condition for session completion, not optional housekeeping, but part of the "definition of done."
• All five dimensions are non-negotiable (build, tests, progress, artifacts, startup); each must be explicitly checked.
• Quality documents make codebase health trackable. You can only proactively fix what you know is degrading.
• Periodically simplify the harness: as model capabilities improve, remove constraints that are no longer needed.
• "Clean up later" equals never cleaning up. Entropy growth is the default state; only active cleanup counteracts it.

Example Files

Copy-ready course examples referenced in this lecture.

benchmark-comparison-template.md

A two-column template for the toggle-off-and-benchmark discipline, the two interpretation questions at the bottom (changed the result vs changed the cost) are what keep the comparison honest.

# Benchmark Comparison Template

Harness A:

- completion rate
- average retries
- bugs caught before human review

Harness B:

- completion rate
- average retries
- bugs caught before human review

Interpretation:

- Which harness changed the result?
- Which harness changed the cost of getting the result?

benchmark-runner.ts

Runnable benchmark harness over eight tasks with per-task pass criteria and seeded failures (PDF encoding, hallucinated citation, cross-user leak), the fixed task slice you re-run after cleanup or a harness change to detect drift.

/**
 * benchmark-runner.ts
 *
 * Reads a benchmark task definition (JSON array of tasks with pass criteria),
 * "executes" each task, records timing and pass/fail, outputs a comparison
 * report showing which tasks pass and which fail.
 *
 * Run: npx tsx docs/lectures/lecture-12-why-every-session-must-leave-a-clean-state/code/benchmark-runner.ts
 */

// ---------------------------------------------------------------------------
// Types
// ---------------------------------------------------------------------------

interface BenchmarkTask {
  id: string;
  name: string;
  category: string;
  passCriteria: string[];
  expectedDurationMs: number;
  // Simulated actual results
  actualDurationMs: number;
  actualPass: boolean;
  failureReason?: string;
}

interface BenchmarkResult {
  id: string;
  name: string;
  category: string;
  criteriaTotal: number;
  criteriaPassed: number;
  pass: boolean;
  expectedMs: number;
  actualMs: number;
  durationDelta: number;
  failureReason?: string;
}

// ---------------------------------------------------------------------------
// Benchmark task definitions
// ---------------------------------------------------------------------------

const benchmarkTasks: BenchmarkTask[] = [
  {
    id: "bench-001",
    name: "Import markdown document",
    category: "Document Pipeline",
    passCriteria: ["File accepted", "Chunks created", "Metadata stored"],
    expectedDurationMs: 100,
    actualDurationMs: 95,
    actualPass: true,
  },
  {
    id: "bench-002",
    name: "Import PDF document",
    category: "Document Pipeline",
    passCriteria: ["File accepted", "Text extracted", "Chunks created", "Metadata stored"],
    expectedDurationMs: 200,
    actualDurationMs: 310,
    actualPass: false,
    failureReason: "PDF text extraction failed on page 3 -- encoding issue",
  },
  {
    id: "bench-003",
    name: "Ask grounded question",
    category: "Q&A Pipeline",
    passCriteria: ["Query received", "Relevant chunks retrieved", "Answer generated", "Citations present"],
    expectedDurationMs: 500,
    actualDurationMs: 480,
    actualPass: true,
  },
  {
    id: "bench-004",
    name: "Ask question with no relevant docs",
    category: "Q&A Pipeline",
    passCriteria: ["Query received", "Graceful 'no results' message", "No hallucinated citations"],
    expectedDurationMs: 400,
    actualDurationMs: 420,
    actualPass: false,
    failureReason: "Model hallucinated a citation instead of saying 'no results'",
  },
  {
    id: "bench-005",
    name: "Delete imported document",
    category: "Document Pipeline",
    passCriteria: ["Document removed from list", "Chunks removed from index", "No orphan data"],
    expectedDurationMs: 150,
    actualDurationMs: 145,
    actualPass: true,
  },
  {
    id: "bench-006",
    name: "Concurrent user access",
    category: "Security",
    passCriteria: ["Users isolated", "No cross-user data leak", "Performance within SLA"],
    expectedDurationMs: 300,
    actualDurationMs: 850,
    actualPass: false,
    failureReason: "Cross-user data leak detected + response time exceeded SLA (850ms > 500ms)",
  },
  {
    id: "bench-007",
    name: "Session continuity after restart",
    category: "Reliability",
    passCriteria: ["State persisted", "Session recovers", "No data loss"],
    expectedDurationMs: 200,
    actualDurationMs: 180,
    actualPass: true,
  },
  {
    id: "bench-008",
    name: "API rate limiting",
    category: "Security",
    passCriteria: ["Rate limit enforced", "429 response after limit", "Legitimate traffic unaffected"],
    expectedDurationMs: 100,
    actualDurationMs: 100,
    actualPass: true,
  },
];
// … trimmed for the lecture — the pattern continues in the same shape

cleanup-loop.md

The weekly "tracing" cadence as five recurring tasks: scans, grade updates, targeted PRs, and a benchmark re-run to confirm cleanup didn't break anything.

# Cleanup Loop Example

Recurring cleanup tasks:

- scan for stale docs
- scan for structural violations
- update quality grades
- open targeted cleanup PRs
- rerun a fixed benchmark slice after cleanup

cleanup-scanner.ts

Runnable end-of-session scanner: point it at a project and it reports stale artifacts (tmp/bak/log files), TODO/FIXME/HACK markers, and structural violations (missing .gitignore, build output or node_modules inside source), severity-tagged.

/**
 * cleanup-scanner.ts
 *
 * Scans a project directory for stale artifacts, dead code, structural
 * violations, and outputs a cleanup report. Helps enforce the "clean state
 * at end of every session" principle.
 *
 * Usage:
 *   npx tsx docs/lectures/lecture-12.../code/cleanup-scanner.ts [path]
 *   (defaults to current working directory)
 *
 * Run: npx tsx docs/lectures/lecture-12-why-every-session-must-leave-a-clean-state/code/cleanup-scanner.ts
 */

import * as fs from "node:fs";
import * as path from "node:path";

// ---------------------------------------------------------------------------
// Types
// ---------------------------------------------------------------------------

interface ScanResult {
  category: string;
  check: string;
  severity: "critical" | "warning" | "info";
  found: string[];
  description: string;
}

// ---------------------------------------------------------------------------
// Scanner checks
// ---------------------------------------------------------------------------

interface ScannerCheck {
  category: string;
  name: string;
  severity: "critical" | "warning" | "info";
  description: string;
  scan: (dir: string) => string[];
}

function createChecks(): ScannerCheck[] {
  return [
    {
      category: "Stale Artifacts",
      name: "Temporary files (*.tmp, *.bak, *.swp)",
      severity: "warning",
      description: "Leftover temp files from editing or processing",
      scan: (dir) => findFiles(dir, [".tmp", ".bak", ".swp", "~"]),
    },
    {
      category: "Stale Artifacts",
      name: "Debug/log files in source",
      severity: "warning",
      description: "Log files that should not be in the source tree",
      scan: (dir) => {
        const found: string[] = [];
        const logPatterns = [".log"];
        for (const p of findFiles(dir, logPatterns)) {
          // Only flag log files in source directories
          if (p.includes("/src/") || p.includes("/lib/") || p.includes("/app/")) {
            found.push(p);
          }
        }
        return found;
      },
    },
    {
      category: "Dead Code",
      name: "Unused imports (placeholder detection)",
      severity: "info",
      description: "Files that may contain unused code (heuristic scan)",
      scan: (dir) => {
        const found: string[] = [];
        const srcDirs = ["src", "lib", "app"];
        for (const srcDir of srcDirs) {
          const srcPath = path.join(dir, srcDir);
          if (fs.existsSync(srcPath)) {
            scanForPatterns(srcPath, ["TODO:", "FIXME:", "HACK:", "XXX:"], found, dir);
          }
        }
        return found;
      },
    },
    {
      category: "Structural Violations",
      name: "Missing .gitignore",
      severity: "warning",
      description: "No .gitignore file found -- risk of committing build artifacts",
      scan: (dir) => fs.existsSync(path.join(dir, ".gitignore")) ? [] : [".gitignore (MISSING)"],
    },
    {
      category: "Structural Violations",
      name: "Build artifacts in source",
      severity: "critical",
      description: "Compiled output (dist/, build/) mixed with source",
      scan: (dir) => {
        const found: string[] = [];
        // Check if dist/ or build/ exists inside src/
        const dirs = ["src/dist", "src/build", "lib/dist", "lib/build"];
        for (const d of dirs) {
          if (fs.existsSync(path.join(dir, d))) {
            found.push(d + "/");
          }
        }
        return found;
      },
    },
    {
      category: "Structural Violations",
      name: "node_modules in source tree",
      severity: "critical",
      description: "node_modules directory found outside root (nested dependency)",
      scan: (dir) => {
        const found: string[] = [];
        const srcDirs = ["src", "lib", "app", "test", "tests"];
        for (const srcDir of srcDirs) {
          const nmPath = path.join(dir, srcDir, "node_modules");
          if (fs.existsSync(nmPath)) {
            found.push(srcDir + "/node_modules/");
// … trimmed for the lecture — the pattern continues in the same shape

What You Have Now

A handoff note format that makes the next session a cold start with no cost.

Write the template now and fill it at the end of your next real session. The test is the same as always: could a fresh agent, reading only the note, continue at full speed? Anything it would have to rediscover belongs in the format.

Save it as harness/session-handoff.md; the lab's checker looks for that file at that path.

Further Reading

• Clean Code - Robert C. Martin: systematic principles of code cleanliness
• Harness Engineering - OpenAI: reproducibility as a core harness design requirement
• Effective Harnesses - Anthropic: the critical role of clean session exits for long-term reliability
• Harness Design for Long-Running Application Development - Anthropic: source for the sprint-splitting-removal experiment above
• Programs, Life Cycles, and Laws of Software Evolution - Lehman: software evolution laws proving system complexity inevitably grows without active maintenance

Exercises

1. Clean State Checklist: Design a session exit checklist for your codebase covering all five dimensions. Apply it across 5 consecutive sessions and record the number of violations per dimension.

2. Benchmark Comparison: Use a fixed task set with two harness variants (with/without clean state requirements). Compare completion rate, retry count, and defect escape rate.

3. Harness Simplification Practice: Pick one harness component, temporarily disable it, and run benchmark tasks. Compare results with and without it. Decide whether to keep, remove, or replace.

Check your understanding