Lesson 02. Discovery Turns Wishes Into Budgets

A specialty insurer launches a wording assistant for its brokers. The requirement, in the sponsor's words, was "make it accurate," and the team took it seriously: they tuned retrieval for weeks, compared four models, and shipped when the demo answers looked right to everyone in the room. Eleven days after launch a broker pastes an answer into an email to a customer. The answer cites an exclusion clause with a real-looking number that does not exist in the customer's policy. The customer's lawyer finds this out first. In the post-mortem the team discovers that nobody had ever written down how often a wrong clause was acceptable, how anyone would measure it, or who would have signed off on the number. "Accurate" had meant something different to the sponsor, the brokers and the engineers, and every one of them believed the others agreed. Decide now: was this an engineering failure, or a failure of the question that was asked?

Illustrative composite. The insurer, the eleven-day timeline and the fabricated clause are a composite of incidents this course draws on; the pattern of a quality requirement with no number is the common part.

This lesson's deliverable is the second file of your Architecture Dossier. An NFR budget sheet and a use-case qualification scorecard for your case, each budget with a number, a measurement and an owner. Both go into dossier/01-nfr-budgets.md, and both serve the Frame stage of the Frame → Decide → Prove → Run loop, where the charter's exit evidence becomes thresholds a design can be held to.

What is actually going on

A conventional system has requirements; a language-model system has distributions. The same input, on the same day, produces a range of outputs, and the range moves when the model, the prompt, the retrieved documents or the phrasing of the question changes. "Accurate" describes a point. A distribution needs a threshold on a measurement taken over a sample, and until someone names the sample, the measurement and the threshold, the word means whatever each listener needs it to mean. That is why the wording assistant shipped: every stakeholder heard their own number, and no number was ever written down.

The failure has a second layer. The team did measure something: they looked at demo answers and judged them right. But a judgement in a meeting is not a measurement, because it has no fixed sample, no fixed grader and no record. The brokers would have graded the same answers differently, and the post-launch incident was the first time anyone graded a real answer against a real policy. The discovery workshop exists to replace that meeting-room judgement with three things the pilot can be held to: real examples, including ugly ones; a baseline from the client's own systems; and every non-functional requirement restated as a budget.

A budget has three parts and each is a separate question to a separate person. The number comes from the business: how often a wrong clause is tolerable is a question for whoever pays for the consequences. The measurement comes from engineering: what sample, what grader, how often. The owner is the person who will be asked, in a year, why the number was missed. Follow the flow below from the sponsor's wish to the qualified use case and notice where each budget's three parts are produced.

The flow has two exits and both lead to the same place: a use case that leaves with a reason. "Not qualified" is not a rejection of the wish; it names what is missing, and the charter's milestones from the previous lesson are where the missing thing gets collected or measured.

The rule and its condition

Every AI non-functional requirement is a budget with a number, a measurement and an owner. Write budgets for at least these: task accuracy on a golden set, the tolerance for ungrounded claims, latency at the 95th percentile, cost per task and per month, availability, data residency and retention, the human-oversight rule, and auditability. Use the budget form for every requirement the client will hold you to when the requirement can be measured on the client's own examples; when a stakeholder cannot say what would count as a miss, the requirement is not a budget yet, and the use case that depends on it is not qualified. Do not qualify it anyway on the promise that the number will come later. It will not, and the launch will happen without it.

The scorecard sits after the budgets. Score value, feasibility, risk and adoption each from one to five, and qualify a use case only when value is at least four, feasibility at least three, risk at most three and a baseline exists. The thresholds are a starting convention; the discipline is that they are written before the scoring, so a favourite use case cannot be argued past them after the fact.

The discovery workshop

Discovery is a half-day with every stakeholder who owns a decision the system touches, and its output is examples, a baseline and answers to a fixed question set. Run it before writing budgets, not after, because a budget written without the people who own the number is a guess with an owner's name on it. The question set below is the one this course uses; ask every question, record who answered, and mark every question that nobody in the room could answer, because those are the charter's open questions.

# Discovery question set

## The problem and its baseline
1. Which decision or task changes, and who owns it today?
2. What does it cost now in time, money or errors, and how is that measured?
3. Show thirty real examples, including the ugly ones. Who can produce them?
4. What does "good enough" look like as a number?
5. What did earlier automation or AI attempts here fail on?

## When it is wrong
6. When the system is wrong, who notices, how fast, and what is the blast radius?
7. Which actions must a person approve, and within what time?
8. Who could be harmed, and how would they complain?
9. What would make you cancel this project at the pilot gate?

## Data and rights
10. Which data does the task need, where is it, who owns it, what classification?
11. Residency, retention and vendor-training constraints?
12. Which regulations or sector rules apply?

## Platform and people
13. Existing cloud commitments and preferred providers?
14. Identity provider, and how a tool would act on behalf of a user?
15. Expected volume now and in eighteen months?
16. Latency expectation per interaction?
17. Budget ceiling per task and per month?
18. Who runs it after go-live: on-call, evals, prompt changes?
19. Exit plan if the vendor changes price or model?
20. Who is the executive sponsor, and what have they promised, to whom?

Two of the questions do more work than the rest. Question three, the thirty examples, is where wishes meet reality: a client who cannot produce thirty real inputs in a week does not have a use case yet, and the ugly examples are the ones that set the tolerance. Question nine, the cancellation condition, is the one sponsors resist, and the resistance is information.

Worked case

Harbor Mutual's charter from the previous lesson fixes the problem (complex first-party property letters), the users (handlers who sign) and the exit evidence (minutes per letter and citation errors, measured before and in week eight). The discovery workshop now has to turn that exit evidence into thresholds and add the budgets the charter did not mention. In the room: the chief operating officer, the head of claims, the chief information security officer, a senior handler and the CFO's delegate. Their answers, condensed: the handlers want a draft while they read the claim file, so a few seconds is fine but half a minute is not; the CISO will not allow inference outside the EU and UK or vendor retention beyond the contract's abuse-monitoring window; the CFO's delegate wants the cost per letter to sit well below what a handler-minute costs; the head of claims says a wrong wording citation that reaches a customer is a reportable complaint, so the tolerance on citations is near zero, while a clumsy sentence is merely annoying.

Illustrative composite. All figures below are modeled for the case: a fully loaded handler cost of about €52 per hour, a baseline of 18 minutes per complex letter from the case system, roughly 1,200 complex letters a week, and a first-pilot memory of drafts that "invented clause numbers."

Four use cases came up in the workshop, and the sponsor wants all of them. Score them before reading on, and decide now which single budget disqualifies adjuster-visit booking.

The budget that disqualifies booking is human oversight. The head of claims requires every action against an external system to be approved by a handler, which means the assistant can only propose a slot, and a handler who must open the calendar to approve a proposed slot saves almost nothing over booking it. Feasibility fails as well, since nobody in the room knew how the adjuster calendar system exposes availability, but oversight is the budget that kills the value, and it would kill it even if the integration were trivial. The chatbot fails on risk and adoption: risk five because generated text reaches customers with no handler in the loop, adoption two because the customer-service team was not in the room and has not asked for it. Simple-letter templating exits the LLM screen from the previous lesson: the inputs are structured claim fields and the letters already come from templates in four minutes; a model would be a slower template engine.

Place the four on the qualification chart and the verdicts become visible. Value runs upward, feasibility runs rightward, and risk is written into each label because the chart has only two axes.

The chart puts the chatbot in "de-risk data first," which is the honest verdict: the value is real, and a later engagement with labelled customer conversations and a customer-service owner in the room might qualify it. The scorecard's job is not to kill ideas but to say what each one is missing.

Here is the budget sheet for the qualified use case, as it sits in dossier/01-nfr-budgets.md. Every row has all three parts, and the rows that came from the charter's exit evidence say so.

# 01 · NFR budgets · Harbor Mutual · Claims correspondence assistant

Baseline (case system, 2026-09-15 to 2026-09-28): 18 min per complex letter;
citation errors 3.1 per 100 letters on the quality team's fifty-letter audit.

| NFR | Budget | Measurement | Owner | From charter? |
|---|---|---|---|---|
| Time per complex letter | ≤ 11 min in pilot week 8 (≥ 35% below baseline) | Case-system timestamps, same letter types, twelve pilot handlers | Head of claims | Yes, exit evidence |
| Wrong-wording citations | ≤ 1 per 100 drafts before sign-off; 0 reaching a customer | Code check of every citation against the archive version table; handler rejection log | Head of claims | Yes, exit evidence |
| Ungrounded claims in drafts | ≤ 2 per 100 drafts on the golden set | Groundedness judge validated monthly against 50 senior-handler labels | ML lead | No |
| Draft ready (latency) | p95 ≤ 12 s from request to complete draft | Gateway traces, handler-facing path only | Platform lead | No |
| Cost per draft | ≤ €0.15 per draft; ≤ €9,000 per month at pilot volume | Gateway spend tagged by feature, all calls including retries and judge | CFO delegate | No |
| Availability | ≥ 99.5% during handler hours (07:00 to 19:00 local) | Gateway success rate excluding client errors | Platform lead | No |
| Residency and retention | Inference and storage in EU or UK regions only; vendor retention ≤ 30 days, no training on our data | Contract clauses plus endpoint configuration exported and checked quarterly | CISO | No |
| Human oversight | 100% of drafts signed by a handler before any use; no external action by the assistant | Sign-off logs; absence of any write tool in the tool catalogue | Head of claims | Yes, non-goal |
| Auditability | Every draft traced with wording version, prompt version, model version, handler | Trace store retained for the claim file's retention period | CISO | No |

## Scorecard (scored 2026-09-16, before the budgets were filled)
See the table in the workshop notes; thresholds: value ≥ 4, feasibility ≥ 3,
risk ≤ 3, baseline measured.

The cost of doing this the other way is the opening story. Without the citation budget, the pilot would have shipped when drafts looked right, and the first fabricated clause number to reach a customer would have been the measurement. With it, a fabricated citation is caught by a code check before a handler ever sees the draft, and the pilot has a number it can fail against in week one instead of a complaint in week eleven.

Faded case

Skip to the open case if this is familiar.

Lumen's charter from the previous lesson names tier-one ticket resolution as the problem, customer support teams as the users, account changes as a non-goal, and tickets resolved without a human touch on a shadow tenant as the exit evidence. Discovery is a one-hour call with the founder and the engineer, because there is nobody else. Most budgets fall out quickly: resolution accuracy on a thirty-ticket golden set per tenant, a wrong-resolution tolerance set by the founder because she is the one who loses the customer, p95 first-token latency for a chat surface, availability that matches what the customers' own products promise, and US-only hosting with the vendor's standard retention because no customer has asked for more yet.

Illustrative composite. Assume roughly 20% month-on-month growth in ticket volume and a pricing model of a flat fee per seat, so every extra model call comes out of margin.

One budget is yours: cost. The founder says "keep it cheap." Write the cost budget with its number's unit, its measurement and its owner, and say why the unit matters more than the number at this stage.

Open case

Return to the benefits helpdesk from the previous lesson. The director's charter, as you wrote it, keeps eligibility decisions with caseworkers and scopes the assistant to procedural questions with a cited source and a disclosed machine identity. In discovery the director says the accuracy requirement is "it must never tell someone they are eligible when they are not," the caseworkers say they cannot label more than a hundred conversations before the spring, the procurement officer says every vendor needs authorisation before it touches citizen data, and the finance lead has a fixed annual budget with no per-call line. Write the budget sheet and the scorecard, and name the budget that cannot be filled. The quiz's first item grades that naming.

Where this breaks

Budgets written before the thirty examples are seen are guesses, and the guesses run tight. A tolerance of one ungrounded claim per hundred sounds careful until the examples show that a third of the real inputs are ambiguous scans where a human would also hedge. Look at the examples first, set the budget second, and expect to revise the number once after the first eval run, with the owner's signature on the revision.

Quality budgets do not belong in a service-level agreement as guarantees. Availability and latency can carry service credits because you control them; accuracy depends on the client's data and on models you do not control. Contract for quality as a process: the eval gate runs before every release, a regression rolls back within a named time, and a quality report ships monthly. The budget sheet is the internal target; the contract promises the process that defends it.

An owner named as "the team" is no owner. Every row needs a person who will be asked why the number was missed, and if two people share a row, split the row. The same applies to the measurement: "we will check" is not a method. If the row cannot name the sample, the grader and the schedule, it is a wish that has learned to sit in a table.

Key Takeaways

• Restate every AI non-functional requirement as a number, a measurement and an owner.
• Refuse to qualify a use case whose budget cannot be filled; name what is missing.
• Run discovery with the question set and every decision owner in the room.
• Score value, feasibility, risk and adoption against thresholds written before scoring.
• Set the number from the examples and the baseline, never from what the model achieves.
• Contract for quality as a process, and for availability and latency as guarantees.

What You Have Now

An NFR budget sheet and a use-case qualification scorecard for your case, each budget with a number, a measurement and an owner. Both live in dossier/01-nfr-budgets.md, immediately after the charter, and the rows that came from the charter's exit evidence say so. Every Decide-stage file reads this sheet: the model-selection record filters candidates by residency and latency before capability, the context specification is bounded by the cost row, and the evaluation plan turns the accuracy and groundedness rows into a golden set and a gate. It serves the Frame stage of the Frame → Decide → Prove → Run loop.

Example Files

The blank budget sheet. Copy it to dossier/01-nfr-budgets.md; a row with any of its three parts missing is a discovery question, not a budget.

# 01 · NFR budgets · <client> · <system>

Baseline (<source system>, <dates>): <measure and number, or "not yet measured:
milestone N">

| NFR | Budget | Measurement | Owner | From charter? |
|---|---|---|---|---|
| Task accuracy on the golden set | ≥ <n>% pass on <k> cases | <eval schedule; grader; how the judge is validated> | <person> | <yes/no> |
| Ungrounded claims | ≤ <n> per 100 sampled answers | <groundedness check; sample rate; human validation cadence> | <person> | |
| Harmful or policy-violating output | <n> criticals; ≤ <n>% flags | <classifier plus human review of flags> | <person> | |
| Latency | p95 first token ≤ <n> s; p95 complete ≤ <n> s | <trace source; which path> | <person> | |
| Cost | ≤ <n> per task; ≤ <n> per month | <gateway spend by feature tag; every call counted> | <person> | |
| Availability | ≥ <n>% over <window> | <gateway success rate; exclusions> | <person> | |
| Residency and retention | <regions>; vendor retention ≤ <n> days; <training prohibition> | <contract clause; configuration export; audit cadence> | <person> | |
| Human oversight | <which outputs or actions, approved by whom, within what time> | <approval logs; share auto-approved> | <person> | |
| Auditability | <what every call is traced with> | <trace store; retention> | <person> | |
| Fairness (if the system faces people) | <gap tolerance across named groups> | <counterfactual subset; cadence> | <person> | |

# Use-case qualification scorecard (thresholds fixed before scoring)

Thresholds: value ≥ 4, feasibility ≥ 3, risk ≤ 3, adoption noted, baseline measured.

| Use case | Value | Feasibility | Risk | Adoption | Baseline | Verdict | What is missing |
|---|---|---|---|---|---|---|---|
| <name> | | | | | | | |

Scoring prompts:
- Value: which P&L line or target moves, by how much, and who owns it?
- Feasibility: data available with rights; thirty real examples pass a first test; integration points known.
- Risk: consequential decision; regulated data; public-facing; irreversible actions.
- Adoption: whose workflow changes; are they in the room; what is their incentive?

Further Reading

• AWS Prescriptive Guidance, architecting a generative-AI proof of concept: hypothesis, success criteria and evidence as the shape of a proof of concept, from a vendor's own lifecycle guidance.
• MIT NANDA, The GenAI Divide: State of AI in Business 2025: the report whose missing-baseline critique is the strongest argument for the baseline row on your sheet.
• OpenTelemetry semantic conventions for generative AI: the attribute names your latency and cost measurements will be recorded under once the system exists.
• The FinOps Framework, 2025 edition: the vocabulary finance teams use for the cost row, including showback and unit economics.

Exercises

1. Take the opening story and write the three budgets that would have caught the fabricated clause: the citation check, the groundedness tolerance and the oversight rule. For each, name the person at a specialty insurer who owns the number.
2. Run the question set against a use case in your own organisation, alone, answering only what you know for certain. Count the unanswered questions; those are the invitation list for the workshop.
3. Fill the budget sheet and the scorecard for your own case from the previous lesson's charter. Mark every row whose number you set without an example or a baseline; those numbers are provisional until the first eval run, and the sheet should say so.

Check your understanding