Autopilot wrapp + a gate correctness fix - #11
Open
sameeeeeeep wants to merge 5 commits into
Open
Conversation
… fix Autopilot ships as a standalone wrapp: lend it a brand/project/idea and it drafts the operating slate cold — voice, ad angle, channel, next move. Choosing ripples downstream; nothing irreversible happens without you. Resolved first: separate wrapp, NOT a third brandbrain studio. The roadmap's "three verbs" argument was retro-justification — the separation had entered the record as a scope guard, and a studio was never evaluated. It was. brandbrain's studio engine is finite by construction (seq.every(id => locks[id])) with no concept of a recurring decision; the cockpit is the part that does NOT generalise (compileTasks is a hand-written if-chain, and the !active.path gate already stops ideabrain reaching the OS); and the OS is one-brand-at-a-time, which contradicts a portfolio at the state level. contextKinds is ["brand","project","idea"], not ["brand"] — the latter re-narrows a wrapp meant to generalise past D2C and breaks the idea→run graduation. Also took brandbrain's auto|approve|manual taxonomy as a concept, not code. It is what makes "autonomous" honest. THE BIGGER RESULT — a live race in 21 wrapps, previously masked by the harness. onReady fires twice (mountConnect + the returning-user probe). With storage that really returns what you wrote, the second pass re-reads the run the first just saved and REPLACES the in-memory object the running pipeline holds, so the pipeline completes into a detached orphan and the UI sits on a run that never finishes. The harness had been hiding this since 2026-07-18 by discarding every write. Backing the mock with a real per-page store turned reel/marquee/arcade red immediately. Fixed in all 21 wrapps and in the wrapp template so new ones inherit it (cartridge.js already had it; it was never propagated). Verified: - harness 70/70 (was 68 runs → 64/2/2), now WITH real storage - examples/apps/proof/run-autopilot-storage.mjs — 19/19 against a real daemon: one company = one file, editing A never rewrites B, torn records skipped, traversal ids refused, per-origin isolation (npm run try-autopilot) - interactions driven end-to-end: cold open, choose, 2-level ripple, unchoose, take-back-inherited, escape hatch, multi-company, budget tiers, error + recovery, responsive at 375/1440 Token figure in the catalog stays dev-reported — live measurement needs Claude Code signed in on this machine. Co-Authored-By: Claude <noreply@anthropic.com>
The claude-code backend read only `usage.input_tokens`, ignoring `cache_creation_input_tokens` and `cache_read_input_tokens`. Measured on a real call: 451 reported vs 34,585 actually consumed. That number is not cosmetic. server.ts feeds it straight into `gate.recordCompletion()`, so every per-origin tokens/day budget was enforcing ~77x more than the consent card promised — and every wrapp surface showing "tokens spent" understated by two orders of magnitude. Cached input is still input: it is context the model processed and the account was billed for. The OpenAI-shaped backends were already correct (`prompt_tokens` is a total). Paired with the correction, the default budget goes 200K -> 2M/day. One decision-sized call measures ~26K tokens, so the old default was ~7 calls a day; a wrapp that cold-opens would have burned a third of it before the user typed anything. The gate is meant to stop runaway origins, not ordinary use. Co-Authored-By: Claude <noreply@anthropic.com>
Pricing. The Autopilot page promised "Free, and there is no second tier"
while TOKENS.md documents a Pro subscription paying developers 75%. Rewritten
to the line CLOUD.md already states: your infra is free, our cloud is Pro —
we charge for infrastructure we run, never for AI access. Team Mode over your
own LAN or a self-hosted relay stays free; the hosted relay and encrypted
backup are Pro.
That rule then contradicted the catalogue, where 12 wrapps advertised a
different Pro made of power-user batch features ("batch shot lists",
"publish to a domain", "whole-site crawls") that cost us nothing to run and
which the stated principle makes free. All 13 pro lists are now the same
shape: a shared board plus encrypted backup.
Honesty. The hero mock showed three staged moves the product cannot generate
(outreach to 32 outfitters, calling a co-packer). movesFor() emits exactly
three shapes; the mock now uses them. Fabricating capability on a page whose
pitch is "revenue would need a connected store, so it says not connected"
undercuts the only thing being sold.
Redline pass (its real audit, run on the page): deduplicated the hero /
feature-01 / free-bullet triple, replaced the unbacked "Your data stays
yours" chip with two claims that are backed, and cut "Free core, forever"
from 4 uses to 1. Redline's "glossary card in a pricing slot" finding is
resolved by the slot now holding an actual plan.
Co-Authored-By: Claude <noreply@anthropic.com>
The harness only ever said yes. grant() approved everything asked for, health
was always green, and every tool call succeeded — so the entire class of "the
user declined" or "the daemon is down" was untested across the catalogue,
despite the wrapp skill explicitly warning that a wrapp must "read the
returned grant and handle partial approval".
Adds ?refuse=<mode>, opt-in per run so the default sweep is byte-identical:
partial only the first tool and first model survive the consent card
tools every tool unticked, models still granted
connect the user closes the card
asleep daemon installed but unreachable — fails at the TRANSPORT, not
just in the health probe (flipping health alone let wrapps sail
straight past it; Autopilot drafted a full slate against a daemon
reporting "not running", which tested nothing)
toolcall full grant, but every tool INVOCATION fails at run time
Swept all 34 wrapps under `asleep`: 34/34 degrade cleanly — zero errors, no
hangs, every one falls back to a sensible pre-connect state. No bug found,
which is the point of running it.
Recorded while doing this: `partial` and `tools` cannot yet find anything,
because the harness's baseline grant is built from an EMPTY scope
(`GRANT = grant({})`) and the returning-user path never calls connect. So
every wrapp already runs with zero granted tools and the full default model
list, and the mock's stream ignores grants entirely. Making partial-approval
testable means seeding the grant from each wrapp's declared scope first —
a bigger change than a flag, and the real blocker.
Co-Authored-By: Claude <noreply@anthropic.com>
Cheap 80% of "seed real grants in the harness", which was the expensive fix I reached for first. The harness cannot see a scope mistake at all: its grant is fabricated from an empty scope and the responder ignores grants entirely. Rather than re-architect that, read each wrapp's own source and compare what it CALLS against what it DECLARES. Catches the two documented gotchas: a tool used but not granted (Higgsfield's whole-connector wildcard, WebSearch, WebFetch), and a model called that the scope does not list (grants are exact-match). Deliberately conservative. genImage() defined-but-never-called is template plumbing, not a finding — several wrapps carry the helper unused, and counting those would have produced a false positive on arcade, which is exactly what a cruder grep did before this was written. Result on the current catalogue: 42 sources, zero findings — every wrapp already declares what it calls. Verified non-vacuous by injecting both failure kinds into autopilot and confirming both fire. This is a "you will be denied in production" check, never a security claim: the gate is default-deny and enforced daemon-side, so these already fail loudly on a developer's first real run. The lint just moves discovery to build time. Not chained into `npm run build` on purpose — a lint that can abort a 36-entry esbuild run is how builds get bypassed. Run it with `npm run lint:scope`. Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The headline is not the wrapp
Chasing Autopilot's token surface turned up a correctness bug in the gate. The claude-code backend counted only
usage.input_tokensand ignoredcache_creation_input_tokens/cache_read_input_tokens. Measured on a real call: 451 reported, 34,585 actually consumed.server.tsfeeds that straight intogate.recordCompletion(), so every per-origin tokens/day budget was ~77x more permissive than the consent card promised. Budgets are an advertised protection; they were not enforcing what they claimed. Default raised 200K → 2M/day to match honest counting (one decision-sized call is ~26K, so the old default was ~7 calls/day).Autopilot
The operating cockpit: lend it a brand / project / idea and it drafts the whole slate — voice, ad angle, channel, next move — with zero input. Choosing ripples downstream. Moves are staged
auto | approve | manual, a taxonomy taken from brandbrain's OS as a concept, not code.Resolved first: separate wrapp, not a third brandbrain studio. The roadmap's "three verbs" argument was retro-justification — the separation had entered the record as a scope guard, and the studio option was never evaluated. It was: brandbrain's studio engine is finite by construction (
seq.every(id => locks[id])), the cockpit is exactly the part that does not generalise (ideabrain already can't reach the OS), and the OS is one-brand-at-a-time. Full reasoning in ROADMAP.md.contextKindsis["brand","project","idea"], not["brand"]— the narrow version would have broken the idea→run graduation.The catalog-wide bug this shook out
Making the harness storage stub real (it discarded every write) turned reel/marquee/arcade red immediately. The double-
onReadyrace was live in 21 wrapps, invisible because the fixture lied. All 21 + the template now carryhydrated.Pricing made consistent
The page promised "no second tier" while TOKENS.md documents a Pro paying developers 75%. Rewritten to CLOUD.md's stated line — your infra is free, our cloud is Pro; we charge for infrastructure we run, never for AI access. That rule then contradicted 12 catalog entries selling power-user batch features as Pro; all 13 are now one shape.
The hero mock also showed three moves the product cannot generate. It now uses
movesFor()'s actual output.Verification
examples/apps/proof/run-autopilot-storage.mjs— 19/19 against a real daemon on a throwawayRELAY_DIR: one company = one file, editing A never rewrites B, torn records skipped, traversal ids refused, per-origin isolationKnown gaps
tokens: 420_000in the catalog is still dev-reported (it's build cost, not runtime).projectcontext is the obvious next upgrade.🤖 Generated with Claude Code