Skip to content

feat(api): let a model write the case, without letting it decide anything - #15

Merged
Madeuss merged 2 commits into
mainfrom
feat/llm-veneer
Aug 30, 2026
Merged

feat(api): let a model write the case, without letting it decide anything#15
Madeuss merged 2 commits into
mainfrom
feat/llm-veneer

Conversation

@Madeuss

@Madeuss Madeuss commented Aug 30, 2026

Copy link
Copy Markdown
Owner

Closes E1 (#5). The last piece of phase 1, and the first code in the repository
that calls a model.

The two boundaries

write() takes Case, never CaseWithSolution. Same discipline as the
solver, different reason: a writer who knows the culprit telegraphs them. Not
deliberately — the guilty character just ends up with the sharper description.

The test asserts the stronger property rather than the obvious one:

for suspect in case.suspects:
    assert user.count(suspect.name) == 1   # only in the cast list
    assert user.count(suspect.id) == 1

Nothing in the prompt distinguishes one suspect from another. A model that
cannot tell them apart cannot write the guilty one as guiltier.

The prompt only ever contains public facts. The veneer has no use for a
restricted fact, so it never receives one — and therefore cannot leak one. Which
means the canary check on the output is not pointed at the model. It is pointed
at us: a canary here proves context assembly upstream is broken (RN-010,
RN-012).

What the model is allowed to produce

A narrow schema: role_title, appearance, manner, and a scene. There is
no field for evidence, for a location, for a time, or for a culprit. Anything
outside those fields cannot reach the game because there is nowhere to put it.

What the schema can't prevent, validation rejects:

Check Why it exists
canary in output RN-012 — and it means an upstream bug, not a bad sentence
invented / dropped / duplicated character the cast is not the model's to edit
one suspect described with another's name survives a read-through, then contradicts every alibi
blank or runaway field a long field is a model that started narrating the case

A rejected draft is discarded, never repaired. A model that broke the cast
list once will break it differently on a patch, and a half-corrected veneer is
harder to reason about than none.

Failure is not fatal

No key, a refusal, a timeout, a rejected draft — the case still prints, with a
note on stderr:

$ make case SEED=42 VENEER=1
veneer skipped: the model call failed: Could not resolve authentication method...
Caso 42 (gerador v2, pt-BR)
...

Prose is a luxury. The mystery is not.

Testing without spending anything

63 tests, none of which need an API key. The model is a stub that returns
exactly the drafts that must not reach a player: the invented character, the
swapped name, the leaked canary, the refusal. Waiting for a real model to emit
one of those by chance is not a test strategy.

Two decisions worth flagging

Model. claude-opus-5, switchable through MANSAO_VENEER_MODEL because it
is the one knob that moves cost per case. I have not measured the cost — there
is no key on this machine — but it is one call per case, and adaptive thinking
means output tokens dominate. Worth measuring against the R$ 0,50 per match
budget before phase 2 adds six NPCs with a turn each.

No server-side refusal fallback. The usual default for Opus 5 code is to
pass fallbacks, so a policy decline reruns on another model inside the same
call. Skipped here deliberately: the veneer is optional, so degrading to no
prose is both simpler and more honest than silently switching models on a
creative-writing task. A refusal on a cosy murder mystery would be interesting
information, and swallowing it would hide that.

Also in this PR

The ADR-0006 rename that phase 0 left behind — ConfiguracaoSettings, and
/health now answers version / environment. openapi.json regenerated.

63 passed · ruff clean · mypy strict: 16 files, no issues

…hing

Closes E1. The generator produces a structure the solver has approved; this
turns it into prose a player would want to read.

Two boundaries define the module, and they are the whole design.

It takes `Case`, never `CaseWithSolution` — the same signature discipline as the
solver, for a different reason. A writer who knew the culprit would telegraph
it. Nobody would mean to; the guilty character would simply end up with the
sharper description, and the mystery would be over before the first question. A
test asserts the stronger property: every suspect appears exactly once in the
prompt, so nothing in the text distinguishes them.

It is only ever shown public facts. The veneer has no use for a restricted one,
so it never receives one and cannot leak one. The canary check on the output is
therefore not aimed at the model — it is aimed at us, because a canary in this
output means context assembly upstream is broken (RN-010, RN-012).

The output schema is narrow on purpose: role, appearance, manner, and a scene.
There is no field in which to invent evidence, move somebody through the house,
or name a killer. What the schema cannot prevent, validation rejects: an
invented or dropped character, a duplicated id, one suspect described with
another's name, a blank or runaway field. A rejected draft is discarded rather
than repaired — a model that broke the cast once will break it differently on a
patch.

Failure is not fatal. No key, a refusal, a network error, a rejected draft: the
case still prints, with a note on stderr. Prose is a luxury; the mystery is not.

The prompt is a versioned file in prompts/veneer/, never a string literal, and
the tests read it rather than restating it.

Also finishes the ADR-0006 rename that phase 0 left behind: Configuracao became
Settings, and the health endpoint now answers with version and environment.

63 tests, all of them keyless — the model is a stub returning exactly the drafts
that must not reach a player. Waiting for a real model to produce one of those
by chance is not a test strategy.
The veneer is short, structured, and its failure modes — an invented character,
a swapped name, a leaked canary — are caught by validation rather than by model
quality. The prompt measures ~400 input tokens and the reply ~400 output, which
puts Haiku 4.5 at roughly US$ 0.0025 per case: about 400 cases per dollar.

Opus would be ~5x that on paper and more in practice, because adaptive thinking
is on by default there and thinking tokens bill as output. That headroom is
better spent in phase 2, where six NPCs each answer a turn against the R$ 0,50
per-match budget.

Env-switchable through MANSAO_VENEER_MODEL, so comparing the two on the same
seed costs about a cent.
@Madeuss
Madeuss merged commit be23050 into main Aug 30, 2026
4 checks passed
@Madeuss
Madeuss deleted the feat/llm-veneer branch August 30, 2026 21:53
Madeuss added a commit that referenced this pull request Aug 30, 2026
The branch carried the original veneer commits while main carried them squashed
— divergent history over identical content, so every renamed file conflicted
falsely. The branch tree already contains everything main has plus the rename,
which is what makes -s ours the honest resolution here rather than a shortcut.
Madeuss added a commit that referenced this pull request Aug 30, 2026
Stacked on #15 — merge that one first and GitHub retargets this to
`main`
automatically.

## Three names, one thing

| Was | Now |
|---|---|
| `firenze` (repository) | `firenze` |
| Mansão (product, docs, README) | Firenze |
| `mansao` (Python package, env prefix, database) | `firenze` |

On a public repository the first name a visitor meets is the
repository's, and
it named nothing in the game. The plan also drops its `(working title)`
marker,
since the name is now decided.

## What moved

Package and imports, the `firenze` CLI command, `MANSAO_` → `FIRENZE_`,
the
compose project, the database user and name, the FastAPI title, and
every path
cited in the documents and the ADRs.

```
63 passed · ruff clean · mypy strict: 16 files, no issues
firenze generate --seed 42 --locale en   # same case, same culprit
```

The case is byte-identical before and after: the rename touches no
constant the
generator seeds from.

## Local cleanup, once

Containers and the pgvector volume were created under the old project
name:

```bash
docker compose -f infra/compose/docker-compose.yml -p mansao down -v
make dev
```

Nothing of value is lost — no case is persisted yet and there are no
migrations.

## Still open, deliberately

The generator only knows how to build a manor, and the veneer prompt
still says
"a Brazilian manor house in the 1930s". Whether the setting follows the
name to
Florence is a question about content, and it belongs in its own change.
@Madeuss Madeuss linked an issue Aug 30, 2026 that may be closed by this pull request
@Madeuss Madeuss added this to the Phase 1 — Case generator milestone Aug 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

E1 — Generate a playable case

1 participant