feat(api): put any model provider behind one port - #18
Merged
Conversation
The veneer called the Anthropic SDK directly. That was right for getting it working and wrong to keep, because the provider is not decided — some API will be called, which one is open. Everything a model can be asked for now goes through one method: a system prompt, a user prompt, a schema, and back an instance of that schema. Two failures kept apart, because a refusal is a fact about the request that belongs in the evals, while unavailability is plumbing. The coupling worth avoiding was never the import. It is the vocabulary that would spread through everything built between now and the decision — a stance machine that knows what a stop_reason is, retry logic shaped by one vendor's error taxonomy. None of that announces itself until it has to be undone, so there is a test that fails when any module outside firenze.model imports a provider SDK. FakeModel fills any schema with deterministic, obviously-synthetic text, so the game can be built and a front end developed with no key and no bill. It fits schemas, never cases: domain validation rejects a fake veneer because the cast it invents belongs to no mystery. That rejection is the line between "this pipeline runs offline" and "this produces something a player could be shown". The provider defaults to none and raises rather than picking one, and model_name no longer defaults to a vendor's model. A default would be the decision being made quietly by whoever set it. This is affordable because of what the project already decided: nothing in the deduction path asks a model for anything (RN-023, RN-032). A model writes prose and proposes a stance; both come back as a validated schema.
Replaces the Anthropic adapter, which was dead code the moment the provider question was raised: an adapter for a provider this project will not call, carrying an SDK dependency for calls that will never happen. It was kept because moving working code behind the new port was easier than deleting it, which is not a reason. Prosa exposes an OpenAI-compatible API, so what replaces it is not a Prosa adapter. It is an OpenAI-compatible adapter with a base URL, which reaches Groq, Together, OpenRouter or a local vLLM by configuration rather than by code. Structured output is the part the documentation does not answer, so the adapter answers it at runtime: a server-enforced JSON schema, then JSON mode with the schema in the prompt, then a plain request with the JSON pulled out of whatever comes back — keeping whichever works. All three failing is a failure. A response that does not validate is discarded, never repaired (RN-022). ADR-0008 records the decision and what it costs. The grounds are credits already held, infrastructure already on Magalu, a monthly ceiling in the currency the eval plan is written in, and supporting a Brazilian cloud — not the eval evidence ADR-0007 said the decision would wait for, and the ADR says so rather than pretending the evidence arrived early. The cost is stated plainly: the catalog is open weights, and those models are weakest exactly where phase 3 is hardest — injection resistance and persona consistency, in Portuguese, which ADR-0005 already flagged as the harder language. What does not degrade is the canary gate: the model is never given a secret fact, and the filter is code. Provider defaults to none until the pilot opens.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Groundwork for phase 2. ADR-0007
draws the boundary; ADR-0008
picks what sits behind it.
The interface
A system prompt, a user prompt, a schema; an instance of it, or a failure. Two
failures kept apart:
ModelUnavailableandModelRefused— a refusal is a factabout the request and belongs in the evals, not buried in generic plumbing.
Affordable because of decisions already made. Nothing in the deduction path
asks a model for anything: solver, verdict, scoring and contradiction detection
are code (RN-023, RN-032). An application whose business logic ran through tool
calling could not draw the line this tightly.
The provider
Magalu Prosa, which exposes an OpenAI-compatible API — so the adapter is not
a Prosa adapter. It is an OpenAI-compatible adapter with a base URL, and Groq,
Together, OpenRouter or a local vLLM are configuration away.
The grounds are in ADR-0008 and are not the ones ADR-0007 said it would wait
for: credits already held, infrastructure already on Magalu, a monthly ceiling
in R$ (the currency the eval plan is written in), and supporting a Brazilian
cloud. The ADR says that plainly rather than pretending the eval evidence
arrived early — and records why the port is what makes deciding early safe.
Structured output, which the docs do not cover
Prosa's documentation does not say whether the gateway enforces schemas. So the
adapter finds out at runtime and remembers:
response_formatwith a JSON schema — the server enforces the shapeAll three failing is a failure. A response that does not validate is discarded,
never repaired (RN-022).
What was removed, and why it should not have been there
The Anthropic adapter. It became dead code the moment the provider question was
raised — an adapter for a provider this project will not call, carrying an SDK
dependency for calls that will never happen. It survived because moving working
code behind the new port was easier than deleting it, which is not a reason.
git logkeeps it.The cost, stated plainly
Prosa's catalog is open weights — Google, Meta, NVIDIA, Qwen — and those models
are weakest exactly where phase 3 is hardest: injection resistance ≥ 95% and
persona consistency, in Portuguese, which ADR-0005 already flagged as the harder
language.
What does not degrade: the canary gate stays 0%. The model is never given a
secret fact, and the filter is code (RN-010, RN-012). A weaker model costs
quality, and quality is what the eval suite exists to measure.
Boundary
A test fails when any module outside
firenze.modelimports a provider SDK. Itcaught two files on its first run — one false positive (naming a provider as a
config value is not coupling, so it now looks for the import) and one real (a
config default quietly naming a vendor's model).
Provider defaults to
noneuntil the pilot opens;fakecovers development.