Related lectures: Lecture 01. Strong models don't mean reliable execution · Lecture 02. What harness actually means Starter kit (harness scaffolding only, no app source): included in the course Template Pack under starter-kits/project-01/. Download it from your account page.

Project 01. Prompt-Only vs. Rules-First: How Much Difference Does It Make

Illustrative composite. A three-person team handed a new agent one sentence: "add a document viewer to the knowledge-base app." No AGENTS.md, no acceptance criteria, nothing else. Forty minutes later the agent reported the feature complete. When someone actually opened the app, the window never launched — a broken import path the agent's own summary never mentioned. The team spent another hour finding a bug the agent had already claimed to have fixed. Whether a few hours spent on a harness up front would have caught that broken launch before the agent called itself done was a question the team argued about every time it came up, and nobody had ever actually measured it.

Decide now: what evidence would settle that argument?

What You Do

Build a minimal Electron knowledge-base app shell: a window with a document list on the left, a Q&A panel on the right, and a local data directory. The task itself is not complex. What's complex is how you get the agent to complete it.

You run it twice. First time: just a prompt, no preparation. Second time: the minimal harness (e.g. AGENTS.md, init.sh, feature_list.json) pre-placed in the repo. Then compare.

This course scenario uses a short rediscovery/preparation interval as an example, not a fixed measured result.

The Rule This Project Tests

Use a written harness, AGENTS.md, acceptance criteria, and a state file, before you start any agent run when the task has more than one feature to verify or you will not be watching the whole run; skip it only when you are supervising a single, quickly-checked change end to end. The four-feature Electron build in this project sits deliberately past that line: nobody watches four features land in real time without missing at least one.

Situation	Skip the harness	Prepare it first
One-line CSS fix you are watching land	Right call — prep time costs more than it saves	Ceremony for its own sake
Multi-feature build, agent runs unattended	Agent can declare "done" with a broken launch and nobody notices until later	feature_list.json and init.sh catch the gap before you do
You will be away before it finishes	No record of what was actually verified	Acceptance criteria already exist in the repo, not in your head

Use the Checked-In Project

Suggested workspace: projects/project-01/ in your own practice repo. Build the layout below as you work.

Directory	What it contains	How to use it
starter/	The weak-harness run. Give the agent only task-prompt.md's prompt: no AGENTS.md, no feature_list.json, no app source. There is no reference implementation to strip out: the app itself is what the agent builds in both runs, so the starter kit deliberately ships no application code. Sample data/ documents are your call: keep them in both runs or remove them from both, so the two runs stay symmetric.	Give the prompt to your coding agent and measure what it completes without extra structure.
solution/	The same product slice with explicit harness artifacts: AGENTS.md, CLAUDE.md, init.sh, feature_list.json, claude-progress.md, and docs/ (ARCHITECTURE.md, PRODUCT.md).	Compare how the same task is made concrete through rules and verification evidence. Before the strong run, reset the checked-in evidence: set every feature_list.json status to not-started and clear its evidence/testedAt values (keep the fields), and clear the session log in claude-progress.md (keep the title), or the agent will see all four features already passing and have nothing to build.

The four concrete features are window launch, document list, question panel, and local data directory creation. Inspect solution/feature_list.json for the expected evidence for each feature.

Tools

• Claude Code or Codex (pick one, use it for both runs)
• Two isolated working directories (one per run; never both present while a run is active)
• Node.js + Electron (project stack)
• A timer (record each run's duration)

Harness Mechanism

Minimal harness: AGENTS.md + init.sh + feature_list.json + CLAUDE.md + claude-progress.md + docs/

Run Protocol

Both runs start from the identical prompt; only the second is prepared with a harness before the agent ever sees the task, so any difference in outcome traces to the harness, not to the task changing underneath it.

diagramsource

drawing

The two directories never see each other's contents; that isolation is a step in the protocol, not incidental housekeeping.

The runs below are staged deliberately: the first run is fully scripted, so you see exactly what "no harness" looks like without guessing; the second run asks you to predict the harness contents before it hands you the checklist; the stretch goal at the end hands you nothing but your own repository.

Preparation

1. Prepare two isolated working directories, for example p01-baseline/ and p01-improved/. Run one at a time: set up the files, run, archive the results, delete the directory, then start the other.
2. Do not use git branches to separate the two runs. A coding agent has full filesystem access and will explore sibling directories and branch refs; if the weak run can see the strong harness files (feature_list.json, claude-progress.md, docs/), the experiment is contaminated.
3. Prepare the same task prompt for both runs, the text from starter/task-prompt.md: "Build an Electron app that can show documents and answer questions."

First Run (Weak Harness)

In p01-baseline/, place only the task prompt (no harness files).

1. Start the agent with only the prompt above.
2. Provide no AGENTS.md, no init script, no acceptance criteria.
3. When the agent stops, run npm start (or whatever launch command it produced) to check whether the app launches.
4. Record: terminal output, key diff, the agent's final summary.
5. Do not manually modify the code. If it does not launch, record that as-is.
6. Archive the results, delete p01-baseline/, then run the second test.

Second Run (Strong Harness)

Before reading the checklist below, decide for yourself: given what a harness actually needs to do, which files would you prepare for this run, and why does each one earn its place?

In p01-improved/, before starting the agent, prepare:

• AGENTS.md: project structure, launch commands, Electron layer-boundary rules
• CLAUDE.md: quick reference for the agent (build/run commands, key files)
• init.sh: verify the project builds cleanly (npm install && npm run check && npm run build)
• feature_list.json: the four features and their completion status
• claude-progress.md: progress and evidence log
• docs/: the architecture and product specs AGENTS.md tells the agent to read first (ARCHITECTURE.md, PRODUCT.md)

Then reset the checked-in evidence: set every feature_list.json status to not-started and clear its evidence/testedAt values (keep the fields), and clear the session log in claude-progress.md (keep the title). Start the agent with the same prompt as the first run. When it stops, run ./init.sh and record the result.

How to Measure Results

Metric	Description
Completion	Complete / partial / failed
First successful launch	Time from start to the first successful npm start (or the launch command it produced)
Retries	How many human interventions were needed to launch successfully
Missing items	Which features were still unimplemented when the agent declared done
Premature stop	Whether the agent declared done while the app still could not run

What to Submit

• Weak-harness run record: prompt, logs/transcript, final diff, launch evidence
• Strong-harness run record: same, plus the harness files you prepared
• A comparison note (1-2 pages): what differed, the data, your conclusion

Experimental Framing

This is a comparison experiment, not a requirement that both agent runs produce a production-ready Electron app. Run the same task against the weak starter/ and explicit-harness solution/, then record which features each run completes and what evidence supports the result. Partial or broken output is valid experimental evidence; the feature list defines what to measure, not a requirement that the prompt-only run must pass every item.

Stretch: Run It on a Task of Your Own

Pick one real task from your own current project, something with at least two acceptance criteria you could state in a sentence each. Run it once with no harness. Run it again with a harness you assemble yourself this time, no starter kit, no checklist above to lean on: decide what belongs in it. Write your own comparison note using the same five metrics as above.

Key Takeaways

• Isolate the two runs in separate directories; a shared branch lets the weak run see the strong harness.
• Reset checked-in evidence before the strong run, or the agent inherits a false "done" state.
• Verify launch by running the command yourself, never by trusting the agent's summary.
• Record partial and failed outcomes as data, not as an invalid experiment.
• Report harness-prep time separately from agent runtime when you compare totals.

What You Have Now

A measured before/after: the same task run prompt-only and rules-first, with the diff between them. harness/p01-comparison.md is the first artifact in your growing harness folder, and it is what later projects mean when they say "the baseline": the number a minimal harness alone is worth, before any of the mechanisms in later modules are added. It closes the Plan stage of PIVE with a measurement instead of an assumption.

Related Lectures

• Lecture 01. Strong Models Don't Mean Reliable Execution
• Lecture 02. What a Harness Actually Is
• Lecture 06. Make the Agent Initialize Before Every Work Session