Skip to content

feat(api): put any model provider behind one port - #18

Merged
Madeuss merged 2 commits into
mainfrom
feat/model-provider-port
Sep 2, 2026
Merged

feat(api): put any model provider behind one port#18
Madeuss merged 2 commits into
mainfrom
feat/model-provider-port

Conversation

@Madeuss

@Madeuss Madeuss commented Aug 31, 2026

Copy link
Copy Markdown
Owner

Groundwork for phase 2. ADR-0007
draws the boundary; ADR-0008
picks what sits behind it.

The interface

def complete(self, *, system: str, user: str, schema: type[Schema], max_tokens: int) -> Schema

A system prompt, a user prompt, a schema; an instance of it, or a failure. Two
failures kept apart: ModelUnavailable and ModelRefused — a refusal is a fact
about the request and belongs in the evals, not buried in generic plumbing.

Affordable because of decisions already made. Nothing in the deduction path
asks a model for anything: solver, verdict, scoring and contradiction detection
are code (RN-023, RN-032). An application whose business logic ran through tool
calling could not draw the line this tightly.

The provider

Magalu Prosa, which exposes an OpenAI-compatible API — so the adapter is not
a Prosa adapter. It is an OpenAI-compatible adapter with a base URL, and Groq,
Together, OpenRouter or a local vLLM are configuration away.

The grounds are in ADR-0008 and are not the ones ADR-0007 said it would wait
for: credits already held, infrastructure already on Magalu, a monthly ceiling
in R$ (the currency the eval plan is written in), and supporting a Brazilian
cloud. The ADR says that plainly rather than pretending the eval evidence
arrived early — and records why the port is what makes deciding early safe.

Structured output, which the docs do not cover

Prosa's documentation does not say whether the gateway enforces schemas. So the
adapter finds out at runtime and remembers:

  1. response_format with a JSON schema — the server enforces the shape
  2. JSON mode plus the schema in the prompt — valid JSON guaranteed, shape is the model's problem
  3. plain request, JSON pulled out of whatever comes back

All three failing is a failure. A response that does not validate is discarded,
never repaired
(RN-022).

What was removed, and why it should not have been there

The Anthropic adapter. It became dead code the moment the provider question was
raised — an adapter for a provider this project will not call, carrying an SDK
dependency for calls that will never happen. It survived because moving working
code behind the new port was easier than deleting it, which is not a reason.
git log keeps it.

The cost, stated plainly

Prosa's catalog is open weights — Google, Meta, NVIDIA, Qwen — and those models
are weakest exactly where phase 3 is hardest: injection resistance ≥ 95% and
persona consistency, in Portuguese, which ADR-0005 already flagged as the harder
language.

What does not degrade: the canary gate stays 0%. The model is never given a
secret fact, and the filter is code (RN-010, RN-012). A weaker model costs
quality, and quality is what the eval suite exists to measure.

Boundary

A test fails when any module outside firenze.model imports a provider SDK. It
caught two files on its first run — one false positive (naming a provider as a
config value is not coupling, so it now looks for the import) and one real (a
config default quietly naming a vendor's model).

85 passed · ruff clean · mypy strict: 20 files, no issues

Provider defaults to none until the pilot opens; fake covers development.

The veneer called the Anthropic SDK directly. That was right for getting it
working and wrong to keep, because the provider is not decided — some API will
be called, which one is open.

Everything a model can be asked for now goes through one method: a system
prompt, a user prompt, a schema, and back an instance of that schema. Two
failures kept apart, because a refusal is a fact about the request that belongs
in the evals, while unavailability is plumbing.

The coupling worth avoiding was never the import. It is the vocabulary that
would spread through everything built between now and the decision — a stance
machine that knows what a stop_reason is, retry logic shaped by one vendor's
error taxonomy. None of that announces itself until it has to be undone, so
there is a test that fails when any module outside firenze.model imports a
provider SDK.

FakeModel fills any schema with deterministic, obviously-synthetic text, so the
game can be built and a front end developed with no key and no bill. It fits
schemas, never cases: domain validation rejects a fake veneer because the cast
it invents belongs to no mystery. That rejection is the line between "this
pipeline runs offline" and "this produces something a player could be shown".

The provider defaults to none and raises rather than picking one, and
model_name no longer defaults to a vendor's model. A default would be the
decision being made quietly by whoever set it.

This is affordable because of what the project already decided: nothing in the
deduction path asks a model for anything (RN-023, RN-032). A model writes prose
and proposes a stance; both come back as a validated schema.
Replaces the Anthropic adapter, which was dead code the moment the provider
question was raised: an adapter for a provider this project will not call,
carrying an SDK dependency for calls that will never happen. It was kept because
moving working code behind the new port was easier than deleting it, which is
not a reason.

Prosa exposes an OpenAI-compatible API, so what replaces it is not a Prosa
adapter. It is an OpenAI-compatible adapter with a base URL, which reaches Groq,
Together, OpenRouter or a local vLLM by configuration rather than by code.

Structured output is the part the documentation does not answer, so the adapter
answers it at runtime: a server-enforced JSON schema, then JSON mode with the
schema in the prompt, then a plain request with the JSON pulled out of whatever
comes back — keeping whichever works. All three failing is a failure. A response
that does not validate is discarded, never repaired (RN-022).

ADR-0008 records the decision and what it costs. The grounds are credits already
held, infrastructure already on Magalu, a monthly ceiling in the currency the
eval plan is written in, and supporting a Brazilian cloud — not the eval
evidence ADR-0007 said the decision would wait for, and the ADR says so rather
than pretending the evidence arrived early.

The cost is stated plainly: the catalog is open weights, and those models are
weakest exactly where phase 3 is hardest — injection resistance and persona
consistency, in Portuguese, which ADR-0005 already flagged as the harder
language. What does not degrade is the canary gate: the model is never given a
secret fact, and the filter is code.

Provider defaults to none until the pilot opens.
@Madeuss
Madeuss merged commit f9505a0 into main Sep 2, 2026
4 checks passed
@Madeuss
Madeuss deleted the feat/model-provider-port branch September 2, 2026 00:35
@Madeuss Madeuss added this to the Phase 2 — Single NPC milestone Sep 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant