Never report success for a site a visitor cannot open

[korya]

Problem

Creo told people their website was live when visitors got a 404. A build that produced no home page was reported as finished, offered for publishing, and celebrated — the platform's most damaging failure mode is not breaking, it is lying.

Nothing between the agent and that celebration ever asked whether the artifact was a site. Commit accepted any set of files, success was inferred from the model's stop reason, and publishing checked only that a version existed. The one component that knew better was the serving gateway, which joins index.html onto every site root and 404s when it is absent.

Solution

Make a version impossible to create unless the vertical can serve it, and give the agent a chance to fix its own mistake first. Three layers, outermost first:

harness    model says "done" ─▶ Commit ─┬─ ok ──────────▶ run.completed
                                        └─ unservable ──▶ repair turn (×2) ─▶ honest failure
project    Commit ─▶ Validate ─▶ Quota ─▶ write        ← nothing is written if refused
publish    Publish/Rollback ─▶ Validate ─▶ pointer flip ← for versions minted before the gate

The profile owns the policy. ValidateArtifact is the floor to ValidatePalette's ceiling — one stops a vertical granting more capability than its level allows, the other stops it declaring victory over something a visitor cannot open. The websites vertical requires index.html, present and non-empty. That is not a convention: it is what the gateway resolves a site root to, so a bundle without one is unservable by construction.

The stores enforce policy they do not author. Both gates are injected closures wired in one place, mirroring the existing Quota hook, so project and publish import no policy package. Inside Commit, Validate runs before Quota — servability is a property of the content, capacity a property of the account, and a tenant near their limit mid-repair should hear "there is no home page yet" (which another turn can fix) rather than "you are out of space" (which it cannot). There is a test pinning that order.

The harness asks rather than checks. When the model declares itself finished, the harness calls Commit and branches on the error. Asking the gate rather than duplicating it means the two cannot disagree, and it needs no new workspace API. An unservable artifact buys up to two repair turns; the instruction is written to the log because reconstruct() rebuilds the conversation from the log alone, so an unlogged one would vanish on takeover and the model would simply re-declare itself done. The budget is counted from those logged events for the same reason.

Two paths that used to lie now do not. An exhausted budget commits nothing and fails in plain language. The step-limit path also goes through the gate — "the work so far is saved" is only true when the work is a site, and it was previously said either way.

Before / After

Same scenario in both: a build that never writes a home page.

Before — success over a 404	After — honest failure
"Your site is ready with all eight pages." Status Ready, Publish enabled, preview shows 404 page not found.	One plain sentence, Publish disabled, and the empty state instead of a 404 — because with no version minted, the client's existing logic already does the right thing.

And when the agent can fix it, the user simply gets a website:

Verified against the real bug, not only fixtures

The CSS-only version from the original report was still in a local data directory, so the backstop was tested against it rather than against a mock:

publish the CSS-only version  → 409  "That version of your site doesn't have a home page,
                                      so it can't go online. Ask for the page you want,
                                      then publish again."
publish the good sibling      → 200
rollback (parent is broken)   → 409

A full qwen3.6 build also ran end to end: it produced a real four-file site, published, and served HTTP 200 — the gate does not false-positive on healthy output.

Honest gaps

The repair acknowledgment is currently invisible in the browser — tracked as #18, since web changes are out of scope here.

A real-model early stop could not be forced on demand. The original occurrence was luck. The repair mechanics are covered exhaustively by deterministic fixtures; the live run proves absence of false positives, not presence of a real-world repair.

Other Changes

• A standing guardrail, not just a fix. assertServable fetches the newest version's preview root after a run completes and requires 200. It lives inside waitCompletedAuthed, which every authenticated completion already funnels through, so a scenario cannot opt out by forgetting; the two AC-1 tests wait on an inline predicate and get it explicitly. Verified it bites by recreating the pre-fix world — gate removed, slow-site writing no home page — which fails with "the run completed but the site root serves HTTP 404 — a visitor would see an error page".
• The fake fleet had drifted into modelling an impossible model. slow-site wrote page1.html…page8.html and no home page — an eight-page website nobody could open — and passed the entire suite for four milestones, because nothing asserted servability. Its first page is now index.html; step count, per-step delay and the kill-mid-run window are unchanged, so both AC-1 tests still prove kill → resume → completion. This corrects a premise in the plan document, which had claimed every script wrote a home page; recorded there as a correction because it is stronger evidence for the same conclusion, not a retraction.
• Two adversarial scripts (no-page, repairs-site) plus asks-before-page stay registered permanently, and AGENTS.md now asks that fixtures model outputs a real model could produce.
• repair.started / repair.completed move from specified-but-unemitted to emitted; validators in the component catalog stops being a declaration with nothing behind it.
• PRD open question Composer clips the text you are typing #6 gets a partial answer: missing-page repairs are autonomous and acknowledged with one line naming the delay rather than the artifact. A deliberate softening of P7's "repairs silently" — the repair is silent, the time it cost is not. Recorded as settling one rung of the ladder, not the ladder.
• The refusal copy is pinned by an httptest-level test covering both the publish and rollback paths — it asserts 409, absence of implementation jargon, and that the sentence names the missing home page, rather than exact strings that a comma edit would break. Verified against both regressions it exists to catch: dropping the mapping yields 500, and degrading the copy to "invalid artifact" trips the jargon check.
• No new dependencies; no web/ or internal/webui/dist changes.

Related:

• Platform reports success and offers Publish for a site with no page #4
• Repair acknowledgment is emitted but never rendered in the web client #18 — the acknowledgment's missing client rendering

🤖 Generated with Claude Code