Practice project: Project 03. Multi-session continuity

Lecture 05. Keeping Context Alive Across Sessions

You ask Claude Code to implement a complete feature. It runs for 30 minutes, does most of the work, but context is running low. You start a new session to continue, and discover it doesn't remember what decisions were made last time, why option A was chosen over option B, which files were already modified, or what state the tests are in. It spends 15 minutes re-exploring the project, and might take an inconsistent approach from last time.

This is the real dilemma AI coding agents face in cross-session tasks. This lecture explains why agents "lose the thread" during long tasks, and how structured state persistence lets a new session quickly pick up where the last one left off.

Context Windows: Not Infinite

Context windows are finite. This isn't a problem that model upgrades can solve: even if window sizes grow to 1M tokens, complex tasks will still exhaust them. Agents aren't just generating code; they're understanding codebases, tracking their own decision history, processing tool output, and maintaining conversation context. All this information grows faster than window expansion.

A deeper problem: the information agents produce isn't uniformly important. Intermediate reasoning steps contain the "why" of decisions: why option A was chosen over B, why this library instead of that one, why a particular optimization was skipped. The final output only contains the "what": the code itself. Compaction strategies usually preserve the latter but lose the former. The next session sees the code but doesn't know why it's written that way, and might "optimize" away a deliberate design decision.

Anthropic observed something interesting in their long-running agent research: when agents sense context is running low, they exhibit a "rushed finish" behavior, rushing to finish current work, skipping verification steps, or choosing a simple solution over the optimal one. Anthropic calls this "context anxiety."

Session Continuity Flow

Without state persistence files, every new session has to start from scratch:

With state persistence files, new sessions can pick up quickly:

Core Concepts

• Context windows are finite: No matter what window size is claimed (128K, 200K, 1M), long tasks will eventually exhaust them. After exhaustion, either compaction (losing information) or reset (starting a new session) is required, both lose something.
• State persistence files: Persisted state files that let a new session unambiguously resume where the last one left off. The most basic form includes progress logs, verification records, and next actions.
• Rebuild cost: The time a new session needs to reach an executable state. A good harness can compress rebuild cost from 15 minutes to 3 minutes.
• Drift: The gap between the agent's understanding and the actual state of the code repository. Every session boundary introduces drift; without control, it compounds session after session.
• Context anxiety: A phenomenon observed by Anthropic, agents exhibit rushed finish behavior when approaching context limits, ending tasks early to avoid information loss. At its core, it's an irrational resource anxiety.
• Compaction vs. reset: Compaction summarizes context within the same session (keeps "what," may lose "why"); reset opens a new session rebuilding from persisted state (clean but depends on artifact completeness).

What Happens When Continuity Breaks

The previous session spent significant context budget analyzing three approaches and choosing option B. This session's agent doesn't know about that analysis and might re-decide based on incomplete information, potentially choosing option A. Same information, different conclusion, because the decision-making context is gone.

Even worse is duplicate work. The agent isn't sure whether certain work was already completed and does it again. Or worse, does half of it, discovers a conflict with the existing implementation, and has to rework. Without progress records, the new session has no idea what's already been done.

Over several sessions, the implementation direction may have silently drifted from the original requirements. Each new session has a slightly different understanding of the project goals. Each deviation compounds on the last, and the final result may be far removed from the original intent.

There's also the verification gap. The previous session's verification results (which tests pass, which fail, why they fail) weren't recorded. The new session has to re-run all verification to understand the current state. Every session re-diagnoses from scratch, every time wasting precious context.

Both OpenAI and Anthropic emphasize structured state persistence in their documentation. OpenAI's harness engineering article makes repository knowledge the "system of record", every operation's results should leave traceable evidence in the repo. Anthropic's long-running agents documentation describes the same persistence in practice — a progress file alongside the git history, so a fresh session can see what was recently worked on.

Practical Approaches to State Persistence

Core approach: Treat the agent like an engineer whose short-term memory gets wiped at every session. Before it "clocks out," it must write down critical information so the next "shift" agent can pick up quickly.

Tool 1: Progress file (PROGRESS.md). The most basic state persistence file:

# Project Progress

## Current State

- Latest commit: abc1234 (feat: add user preferences endpoint)
- Test status: 42/43 passing (test_pagination_edge_case failing)
- Lint: passing

## Completed

- [x] User model and database migration
- [x] Basic CRUD endpoints
- [x] Auth middleware integration

## In Progress

- [ ] Pagination feature (90% - edge case test failing)

## Known Issues

- test_pagination_edge_case returns 500 on empty result sets
- Need to confirm whether deleted users should appear in listings

## Next Steps

1. Fix pagination edge case bug
2. Add "include deleted users" query parameter
3. Update API documentation

Tool 2: Decision log (DECISIONS.md). Record important design decisions and reasons. No need for detailed design documents, just "what decision, why, when":

# Design Decisions

## 2024-01-15: Use Redis for user preferences caching

- Reason: High read frequency (every API call), small data size
- Rejected alternative: PostgreSQL materialized view (high change frequency makes maintenance cost not worthwhile)
- Constraint: Cache TTL of 5 minutes, active invalidation on write

Tool 3: Git commits as checkpoints. Commit after completing each atomic unit of work. Commit messages should explain what was done and why. These are free, automatically versioned state snapshots.

Tool 4: init.sh or harness initialization flow. Specify in AGENTS.md the "clock-in" and "clock-out" routines:

## At session start (clock in)

1. Read PROGRESS.md for current state
2. Read DECISIONS.md for important decisions
3. Run make check to confirm repo is in consistent state
4. Read the recent history (`git log --oneline -20`) — what actually changed, not just what was written down
5. Start the app and smoke one feature that was passing in the last session
6. If the smoke shows breakage, repair that first — repair outranks progress
7. Continue from PROGRESS.md "Next Steps" section

## Before session end (clock out)

1. Update PROGRESS.md
2. Run make check to confirm consistent state
3. Commit all completed work

Reading the state files is the floor, not the ritual. A session that starts with file reads alone is still trusting state files over the repo — and state files drift. The smoke converts "the repo is probably fine" into "the repo is verified fine" before any new context is spent, and the precedence matters: building new features on undocumented breakage compounds the damage and makes the regression harder to attribute. No session builds on undocumented breakage.

Mixed strategy: Not every task needs a context reset. Short tasks (under 30 minutes) can complete within one session. Long tasks (spanning sessions) must use progress files and decision logs for continuity. Decision criterion: once a task is well past the halfway point of the window and isn't about to finish, start preparing the handoff.

A Deeper Look at Context Anxiety

Anthropic's March 2026 research further revealed the specific manifestations of context anxiety: on Sonnet 4.5, when context approaches the window limit, the agent shows strong "rushed finish" behavior.

Two strategies address this:

Compaction: Summarizing early conversation within the same session. Advantage: maintains continuity, the agent can see "what." Disadvantage: "why" is often lost in summaries, why option B was chosen over A, why a particular optimization was skipped. More critically, compaction doesn't eliminate context anxiety, the agent knows context was once large, and psychologically still tends to rush to finish.

Context reset: Completely clearing context, opening a new session, rebuilding from persisted artifacts. Advantage: clean mental state, the new session has no "I'm running out of time" anxiety. Disadvantage: depends on the completeness of handoff artifacts. If the progress file is missing critical information, the new session may waste time going in the wrong direction.

Anthropic's actual data: for Sonnet 4.5, context anxiety is severe enough that compaction alone isn't sufficient, context reset becomes a critical component of harness design. But for Opus 4.5, this behavior is greatly diminished, and compaction can manage context without relying on resets. This means: harness design needs specific understanding of the target model, not a one-size-fits-all template.

Source: Anthropic: Harness design for long-running application development

Trace It: The 5-Session Blog

An agent is tasked with a blog system with user authentication, 12 feature points, ~5 sessions of work, no state-persistence files at all.

Predict before reading on: session 1 finishes the user model and basic routes. What happens in session 2, and what does session 5 look like?

Trace it against this lecture's core claim that sessions lose information: session 2 opens with no memory of the auth middleware's interface contract, so it burns time re-inferring intent instead of building. Session 3 can't tell finished work from pending work, so it starts reimplementing features that already exist. By session 5 the repo is full of redundant code and the core auth feature still hasn't passed end-to-end tests.

Now add progress files, decision logs, verification records, and git checkpoints, updated automatically at each session's end. Session 2's rebuild cost collapses: the interface contract is written down, not re-inferred. Completion stops regressing instead of drifting.

Key Takeaways

• Context windows are a finite resource. Long tasks will span sessions, and sessions will lose information: that's objective reality.
• The solution isn't bigger windows. It's better state persistence. Progress files, decision logs, and git checkpoints work together to let new sessions pick up where previous ones left off.
• Treat the agent like an engineer whose short-term memory gets wiped every session: before "clocking out," write down what was done, why, and what's next.
• Rebuild cost is the key metric. A good harness should get new sessions to an executable state within 3 minutes.
• Mixed strategy: short tasks within sessions, long tasks with structured artifacts for continuity.

Example Files

Copy-ready course examples referenced in this lecture.

continuity-checklist.md

Four yes/no questions that test whether your repo survives a session boundary, any "no" is a continuity gap the next session pays for.

# Continuity Checklist

- Can a fresh agent identify recent work in under five minutes?
- Is the current stable startup path documented?
- Is unfinished work clearly identified?
- Is the next best task visible without reading old chat logs?

session-handoff.md

A filled-in handoff at its minimum useful size, note it separates "Completed" from "Broken or Unverified" so the next session never mistakes attempted work for verified work.

# Session Handoff Example

## Completed

- Added markdown import support
- Added a basic document list in the renderer

## Broken or Unverified

- Import succeeds for `.md` but fails for large `.txt` files
- The app starts, but the detail view has not been wired up

## Next Best Step

- Fix `.txt` import path
- Verify import end-to-end
- Then add the document detail panel

session-simulator.ts

Runnable two-session simulation of a six-step task: without a handoff, session B redoes session A's three completed steps; with one, it resumes at step 4, the duplicated-work cost printed side by side.

/**
 * session-simulator.ts
 *
 * Simulates two sessions working on a multi-step task.
 *   Run 1: No handoff file -- Session B starts from scratch and duplicates work.
 *   Run 2: With handoff file -- Session B picks up where Session A left off.
 *
 * Run: npx tsx docs/lectures/lecture-05-why-long-running-tasks-lose-continuity/code/session-simulator.ts
 */

// ---------------------------------------------------------------------------
// Types
// ---------------------------------------------------------------------------

interface TaskStep {
  id: number;
  name: string;
  durationMs: number;
}

interface SessionResult {
  session: string;
  stepsCompleted: number;
  totalDurationMs: number;
  duplicatedSteps: number;
  output: string[];
}

// ---------------------------------------------------------------------------
// Task definition
// ---------------------------------------------------------------------------

const taskSteps: TaskStep[] = [
  { id: 1, name: "Read project structure", durationMs: 50 },
  { id: 2, name: "Understand existing auth module", durationMs: 80 },
  { id: 3, name: "Design new search endpoint", durationMs: 60 },
  { id: 4, name: "Implement search endpoint", durationMs: 100 },
  { id: 5, name: "Write integration tests", durationMs: 70 },
  { id: 6, name: "Update documentation", durationMs: 40 },
];

// ---------------------------------------------------------------------------
// Simulated session executor
// ---------------------------------------------------------------------------

/** Simulate a session doing work. */
function runSession(
  sessionName: string,
  startStep: number,
  endStep: number
): SessionResult {
  const output: string[] = [];
  let totalDuration = 0;

  for (let i = startStep; i <= endStep; i++) {
    const step = taskSteps.find((s) => s.id === i)!;
    totalDuration += step.durationMs;
    output.push(`  [${sessionName}] Step ${step.id}: ${step.name} (${step.durationMs}ms)`);
  }

  return {
    session: sessionName,
    stepsCompleted: endStep - startStep + 1,
    totalDurationMs: totalDuration,
    duplicatedSteps: 0,
    output,
  };
}

// ---------------------------------------------------------------------------
// Run 1: No handoff file
// ---------------------------------------------------------------------------

function simulateNoHandoff(): { sessionA: SessionResult; sessionB: SessionResult } {
  // Session A does steps 1-3 before timing out
  const sessionA = runSession("Session A", 1, 3);

  // Session B has no context, starts from scratch
  const sessionB = runSession("Session B", 1, 6);
  sessionB.duplicatedSteps = 3; // Redoes steps 1-3 that A already did

  return { sessionA, sessionB };
}

// ---------------------------------------------------------------------------
// Run 2: With handoff file
// ---------------------------------------------------------------------------

function simulateWithHandoff(): { sessionA: SessionResult; sessionB: SessionResult } {
  // Session A does steps 1-3 and writes a handoff file
  const sessionA = runSession("Session A", 1, 3);

  // Session B reads handoff, starts from step 4
  const sessionB = runSession("Session B", 4, 6);
  sessionB.duplicatedSteps = 0;

  return { sessionA, sessionB };
}

// ---------------------------------------------------------------------------
// Reporting
// ---------------------------------------------------------------------------

function pad(s: string, len: number): string {
  return s.length >= len ? s : s + " ".repeat(len - s.length);
}

function printRun(title: string, result: { sessionA: SessionResult; sessionB: SessionResult }): void {
  console.log("\n" + "=".repeat(80));
  console.log(`  ${title}`);
  console.log("=".repeat(80));

  console.log("\n  Session A output:");
  for (const line of result.sessionA.output) {
    console.log(line);
  }

  console.log("\n  Session B output:");
  for (const line of result.sessionB.output) {
    console.log(line);
// … trimmed for the lecture — the pattern continues in the same shape

What You Have Now

A continuity file that survives a context reset with no information loss.

Test it the honest way: write the file mid-task, imagine the session dies, and ask whether a fresh agent reading only that file could continue without re-deriving anything. Whatever it would have to re-derive is what your format is still missing.

Save it as harness/continuity.md; the lab's checker looks for that file at that path.

Further Reading

• Anthropic: Effective Harnesses for Long-Running Agents
• OpenAI: Harness Engineering
• Lost in the Middle: How Language Models Use Long Contexts
• Claude Code Documentation
• HumanLayer: Harness Engineering for Coding Agents

Exercises

1. State persistence measurement: Pick a development task needing at least 3 sessions. Without providing any state persistence files, record at each session start how much context the agent spends "figuring out what happened last time." After each session, create a progress file and let the next session start from it. Compare rebuild costs with and without progress files.

2. Handoff template design: Design a minimal handoff template with four fields: repo state (commit hash), runtime state (test pass rate), blockers, next actions. Let a completely fresh agent session restore project state using only this template. Record ambiguities encountered during restoration, iterate to improve the template.

3. Mixed strategy experiment: In a 5-session development task, compare three strategies: (a) always start fresh sessions + progress files, (b) do as much as possible in one session (context compaction), (c) mixed strategy (short tasks in-session, long tasks across sessions + progress files). Compare rebuild time, feature completion rate, and decision consistency.

Check your understanding