Teach QA and the BA two rules, both harvested from live failures

[patelanil]

Two agent files, +142 lines. No behaviour changes elsewhere.

Both rules came out of a real specification-recovery engagement where the agents — and I — got things wrong in ways that looked like success. Each rule carries the worked example that produced it, because the lesson does not land without the detail.

1. QA — the premise question

Asked to review a test-harness spec, QA reviewed the mechanics thoroughly and never asked where the expected outcomes came from. They came from the code under test.

Such a suite cannot fail when the code is wrong. It asserts that the code does what it does. The failure is invisible from inside the test — coverage looks high, assertions look specific, and none of it can ever disagree with the implementation.

QA now asks, before reviewing anything:

How do we know the code did what it should have done?

Then traces every expected outcome to its origin. From a requirement, a design document, or a named person: legitimate. From the code, its output, or a database that code populated: circular, reject it.

Three shapes of the same mistake are named explicitly, because only the first is obvious:

• expectations recorded from a run;
• "coverage" measured by diffing what the code writes against what the tests assert;
• an expected outcome justified by a code citation instead of a rule or a person.

2. QA — the absence question

A sweep reported that a status transition was missing from seed data and the framework therefore throws. It was verified six ways — absent from seed data, absent from upgrade data, absent in a second checkout at a different version, no other component seeds it, the framework does throw, no service overrides it. Every link held.

The finding was still wrong.

The seed data modelled a four-step physical return lifecycle — requested, accepted, received, completed — and recorded that order in a field the framework never reads. The transition was missing because it would let a return complete without the goods ever arriving. The framework was not failing; it was defending the model. The defect was in the code that skipped the ladder, and a sibling flow already walked it correctly.

Seeding the row would have silenced the exception and deleted the control.

So QA now asks, before reporting any absence, whether it is absent by accident or on purpose — looking for ordering fields, a sibling path that honours the constraint, and what the absence prevents. Then it checks the remedy it is implying, because a finding phrased as "X is missing" tells the reader to add X.

Proving that a failure happens is not the same as proving it is a bug.

Also adds reachability: trace callers before reporting, and label unreachable findings LATENT. A confident defect report about dead code costs the reader real time and teaches them to discount the next one.

3. BA — specification recovery

Writing requirements for a system that already exists is legitimate and common. Reading the working code to do it produces a document that looks exactly like a specification and can never disagree with the implementation.

Working code is evidence, never a requirement. It proves a behaviour exists, not that anyone wanted it — and a long-running system accumulates behaviours nobody asked for.

Five rules: elicit before reading, tag every statement with its provenance, treat unowned behaviour as a finding rather than promoting it, separate configuration from requirement, and leave silence silent rather than filling it from code.

With a give-away test for self-checking: if a business person could not have drawn the distinction unprompted, it came from the code.

Evidence these work

The same engagement re-verified 26 findings adversarially under these rules. About half were wrong in a way that changed the action — three refuted outright, twelve reframed, two understated. The absence rule alone caught two cases of code inventing a status the data model deliberately does not have.