Lesson 08. Deployment and Evolution

The agent worked well on your laptop. The heartbeat ran every 15 minutes, the guardrail hook blocked what it should, and every write it wanted to make surfaced as a confirmation prompt you actually saw and answered, because you were sitting there with the lid open. Then you closed the lid to go to a two-day conference, and the heartbeat stopped, not because anything broke, but because the process it ran in went to sleep with the laptop. You came back to two days of silence you mistook for two days of nothing happening, and it took a stale calendar sync to notice the difference.

The deeper problem than "the laptop slept" is what that outage revealed: the whole system had been implicitly relying on you being present. Confirmation prompts assumed a human was watching. "It'll just run in the background" assumed the background kept existing. A standing agent that only stands while your laptop is open isn't standing at all. It's sitting, and it needs an always-on home before any of the guardrail work from Lesson 07 means what you think it means, because unattended is exactly the condition those guardrails were built for.

A container is the other common shape for this: package the heartbeat and its dependencies into an image, run it with a restart policy and a scheduled trigger (a sidecar timer, or the orchestrator's own cron primitive) instead of a systemd unit. The tradeoffs are the ones you'd expect: a container gives you a reproducible, portable image at the cost of an extra abstraction layer between you and the process; a systemd unit gives you direct visibility (journalctl, plain log files, no container runtime to debug) at the cost of being tied to one host's OS. Either is fine. What matters is that whichever you pick, the three properties below (no-human-in-the-loop, single source of truth for config, and a run log you actually read) hold regardless of the wrapper around the process.

What changes when nothing is watching

Move the agent to a small always-on host (a spare machine, a cheap VPS, a home server that doesn't sleep) and one assumption breaks cleanly: there is no longer a human available to approve anything ad hoc. Every write-confirmation gate from Lesson 07 that used to mean "ask the person at the keyboard" now means "there is no keyboard, only the allow-list." If a command isn't on it, it doesn't run, full stop. There's no fallback "I'll just say yes this once" because there's no one there to say it. This is the point at which the allow-list stops being one layer among several and becomes the entire safety story. Anything you didn't put on the list before deployment is a capability the agent simply doesn't have, forever, until you deliberately add it back with the same review discipline as writing it the first time.

This also changes who reviews an allow-list addition. On the laptop, "add this to the allow-list" and "approve this one-off" could be the same five-second decision, made in the moment, by you. Unattended, the two have to be separated in time: the addition happens deliberately, before deployment or during a scheduled review (the monthly evolution review, below), never as a reaction to the agent currently wanting to do something it can't. If you find yourself SSHing into the host to widen the allow-list because the agent is blocked on something right now, that's a signal the review process broke down, not a signal the process worked as intended.

Config and secrets on the host

The host needs its own config, separate from whatever lived in your laptop's shell profile. Three rules keep this from becoming the next incident:

• Secrets load at process start from an env file with restrictive permissions, never baked into a container image or committed anywhere. Same discipline as Lesson 07's secrets handling, now applied to a machine you don't watch as closely. chmod 600 the file and confirm the service's own user owns it, not the account you SSH in with.
• One source of truth for config. A single .env (or equivalent) the deployment unit reads on start, not scattered exports across shell scripts that drift out of sync with each other. If a value needs to change, it changes in exactly one file.
• Restart-safe. Secrets and config are read fresh at boot, not cached in a way that survives a restart with stale values. If you rotate a token (Lesson 07), a restart of the service should be sufficient to pick it up. If it isn't, that's a bug in the deployment unit, not a manual step you have to remember.

Keep a .env.example alongside the real file, checked into version control with placeholder values only. It documents exactly which variables the deployment unit expects without ever containing a live credential. When you rotate a token, diff the real file against the example first: if the example has a key the real file doesn't, that's a config drift you want to catch before the next restart, not after.

Observability: the run log

On the laptop, "did it work" meant watching it happen. Unattended, the run log is the only record. Write it so that reading it answers "did anything go wrong" without re-running anything.

{"ts":"2026-08-19T09:00:03Z","beat":"heartbeat","check":"calendar","status":"ok","findings":0,"ms":142}
{"ts":"2026-08-19T09:00:04Z","beat":"heartbeat","check":"github","status":"ok","findings":2,"ms":890}
{"ts":"2026-08-19T09:00:05Z","beat":"heartbeat","result":"complete","findings_routed":1,"ms":1050}
{"ts":"2026-08-19T09:15:02Z","beat":"heartbeat","check":"calendar","status":"error","error":"timeout","ms":5001}
{"ts":"2026-08-19T09:15:03Z","beat":"guardrail","decision":"BLOCK","command":"git push --force origin main"}

One line per event, structured, timestamped, and specific enough that the first thing you read when something looks off is this file, not a re-run, not a guess. A blocked command, a failed check, and a completed beat with zero findings should all be distinguishable at a glance, because "the agent did nothing" and "the agent tried to do something and got blocked" look identical from the outside if the log doesn't say which.

Where you actually look first, in order: the last beat: complete line (proves the process is alive and finishing runs), then any status: error lines since (proves a check is failing, and which one), then any guardrail lines (proves whether something got proposed and stopped). A one-line grep answers most "is something wrong" questions without opening a shell on the host at all:

tail -n 20 /var/log/standing-agent/run.log | jq -c 'select(.status == "error" or .decision == "BLOCK")'

If that returns nothing across the last 20 lines, the system is healthy even if the ping channel has been quiet. Quiet and broken look the same from your phone, but not from the log.

Backup and recovery of the memory store

Everything else in this system is recreatable: the skills are files you wrote and still have, the guardrail config is a JSON file under version control, the heartbeat script is code. The memory store from Lesson 02 is the one thing that isn't. It's accumulated state, built up beat by beat and session by session, and losing it means losing months of context the agent will never rebuild on its own.

Back it up on a schedule separate from the heartbeat itself (a nightly cron entry is enough for a file-based store), keep at least a few generations rather than overwriting the same backup file every night, and (this is the part almost everyone skips) actually run a restore once, onto a scratch copy, before you need it for real. A backup you've never restored from is a hope, not a backup.

The restore drill costs ten minutes and answers three questions you don't want to be answering for the first time during a real incident: does the archive actually extract cleanly, does the extracted directory have the shape the heartbeat expects, and how long does the whole process take end to end. Run deploy/restore-memory.sh against last night's archive into a throwaway directory, diff it against the live memory store, and only then trust the backup pipeline enough to stop thinking about it.

The monthly review: evolution log

The guardrails from Lesson 07 and the deployment from this lesson are not a one-time setup you walk away from. Once a month, review what the agent got wrong — a false-positive ping, a skill that fired on the wrong trigger, a command it tried that the allow-list correctly blocked but that turned out to be a legitimate need you hadn't anticipated — and turn each one into exactly one of three things: a new or edited rule, a new or edited skill, or a permission removed (or, more rarely, deliberately added). Write the change down in an evolution log so the reasoning survives past the moment you made it.

| Date | Incident | Root cause | Change | Type |
|---|---|---|---|---|
| 2026-08-22 | Ping fired for a routine dependency-bot PR comment | `github` check's threshold treated all "review_requested" reasons as ping-worthy | Narrowed threshold to exclude bot authors | rule |
| 2026-09-03 | `triage-github-issue` fired on a PR review request | Description didn't rule out PRs explicitly | Added negative clause to skill description | skill |
| 2026-09-14 | Allow-list blocked a legitimate `git tag` release step | Tagging was never anticipated at initial deploy | Added `git tag` (exact match) to allow-list after manual review | permission added |

Every row above started as a real incident and ends in exactly one category. The picture below is the same sort, drawn as a decision instead of a table:

Day 1, Day 30, Day 90

Day 1: a minimal allow-list, heavy logging, every write requires confirmation because there's no track record yet to justify otherwise. The agent can do very little, on purpose. The point of day 1 is to observe, not to be maximally useful.

Day 30: a handful of skills have been added from procedures you noticed yourself repeating by hand; one permission has been narrowed after a near-miss the guardrail correctly caught (like the main-branch close call from Lesson 07); the evolution log has real entries with real root causes, not placeholders.

Day 90: the core is stable. Most weeks produce zero evolution-log entries, not because nothing happens but because the system has already absorbed the recurring failure shapes into rules, skills, and a tuned allow-list. The agent does more than it could on day 1, and its blast radius is smaller than day 1's default-open assumptions would have produced, because every permission it has was earned by a specific, logged, reviewed need rather than granted up front "to be safe." That's the compounding effect this whole course has been building toward: trust that accrues from evidence, not from hope.

Key Takeaways

• Moving off the laptop removes the human-in-the-loop assumption. The allow-list stops being one guardrail layer among several and becomes the entire safety story for unattended writes.
• Secrets load fresh at process start from a single source of truth, never baked into an image or scattered across shell profiles.
• The run log is the only record once nothing is watching live. Write it structured enough that silence-as-success and silence-as-outage are distinguishable at a glance.
• The memory store is the one irreplaceable asset. Back it up on its own schedule and prove the restore works before you need it.
• A monthly review turns every real incident into exactly one of: a rule, a skill, or a permission change, logged with its root cause, not just its fix.
• The system should get smaller in blast radius over time even as it gets more capable. Permissions earned by evidence, not granted by default.

What You Have Now

The agent running on a schedule outside your laptop, with an evolution log that closes the loop: a systemd service and timer running the Lesson 06 heartbeat unattended, a structured run-log format you can read at a glance, and an evolution-log template that turns every real incident into a tracked rule, skill, or permission change.

Example Files

deploy/agent-heartbeat.service

The systemd service unit. Runs the heartbeat script once per invocation; the timer below controls the schedule. Install to /etc/systemd/system/agent-heartbeat.service.

[Unit]
Description=Standing agent heartbeat
After=network-online.target
Wants=network-online.target

[Service]
Type=oneshot
User=agent
WorkingDirectory=/opt/standing-agent
EnvironmentFile=/opt/standing-agent/.env
ExecStart=/usr/local/bin/bun run /opt/standing-agent/heartbeat.ts
# Secrets live only in EnvironmentFile, never in this unit file or the image.
StandardOutput=append:/var/log/standing-agent/run.log
StandardError=append:/var/log/standing-agent/run.log

[Install]
WantedBy=multi-user.target

deploy/agent-heartbeat.timer

The schedule. Every 15 minutes, matching the interval used in Lesson 06's examples. Adjust to whatever cadence your checks actually need. Install to /etc/systemd/system/agent-heartbeat.timer.

[Unit]
Description=Run the standing agent heartbeat every 15 minutes

[Timer]
OnBootSec=1min
OnUnitActiveSec=15min
Persistent=true

[Install]
WantedBy=timers.target

Enable and start it:

sudo systemctl daemon-reload
sudo systemctl enable --now agent-heartbeat.timer
sudo systemctl list-timers agent-heartbeat.timer

deploy/backup-memory.sh

The memory backup script, run nightly by its own timer (a second, much shorter systemd timer pair, or a plain cron entry). Keeps the last 14 generations.

#!/usr/bin/env bash
# deploy/backup-memory.sh
# Nightly backup of the memory store (Lesson 02). Run separately from the
# heartbeat -- a corrupted memory store shouldn't also corrupt its own backup.
set -euo pipefail

SRC="/opt/standing-agent/memory"
DEST="/opt/standing-agent/backups"
STAMP="$(date -u +%Y%m%dT%H%M%SZ)"

mkdir -p "$DEST"
tar -czf "$DEST/memory-$STAMP.tar.gz" -C "$(dirname "$SRC")" "$(basename "$SRC")"

# keep the last 14 generations, drop anything older
ls -1t "$DEST"/memory-*.tar.gz | tail -n +15 | xargs -r rm --

echo "backed up memory to $DEST/memory-$STAMP.tar.gz"

.env.example

Checked into version control with placeholder values only. The real .env on the host mirrors these keys with live values and is never committed. chmod 600 the real file.

# .env.example — copy to .env on the host, fill in real values, chmod 600 it.

# GitHub tool wrapper (Lesson 04) — a fine-grained, short-lived token scoped
# to only the repos this agent needs.
GITHUB_TOKEN=

# Heartbeat ping tier (Lesson 06) — webhook URL for the chat surface the
# agent posts to.
PING_WEBHOOK_URL=

# Memory store location (Lesson 02).
MEMORY_DIR=/opt/standing-agent/memory

deploy/restore-memory.sh

The restore drill, run against a backup produced by backup-memory.sh. Always target a scratch directory. Never restore directly over the live memory store without first diffing.

#!/usr/bin/env bash
# deploy/restore-memory.sh <archive> <target-dir>
# Restores a memory backup into a scratch directory for inspection or
# recovery. Never overwrites the live store directly -- diff first.
set -euo pipefail

ARCHIVE="${1:?usage: restore-memory.sh <archive.tar.gz> <target-dir>}"
TARGET="${2:?usage: restore-memory.sh <archive.tar.gz> <target-dir>}"

mkdir -p "$TARGET"
tar -xzf "$ARCHIVE" -C "$TARGET"

echo "restored $ARCHIVE into $TARGET"
echo "diff against the live store before replacing it:"
echo "  diff -rq \"$TARGET/memory\" /opt/standing-agent/memory"

evolution-log.md

The template. Start every project with this file, one row per real incident, never a placeholder row.

# Evolution Log

Every entry below is a real incident this agent produced or caught, and the
change made in response. One row per incident. Category is exactly one of:
rule, skill, or permission.

| Date | Incident | Root cause | Change | Type |
|---|---|---|---|---|

Exercises

1. Write your own deploy/agent-heartbeat.service and .timer pair for the host you actually plan to run on (adjust User, WorkingDirectory, and the interval). Confirm with systemctl status agent-heartbeat.timer that it's scheduled before you walk away from it.
2. Run deploy/backup-memory.sh once, then deploy/restore-memory.sh against the resulting archive into a scratch directory, and diff it against the original. If you skip this step you don't have a backup, you have an assumption.
3. Set up your own .env from .env.example, chmod 600 it, and confirm with ls -l that only the service's user can read it.
4. Start your own evolution-log.md today, even with zero entries yet. In 30 days, come back and fill in at least one real row. If you can't, that's worth noticing too.

Check your understanding