Skip to content

Decide what else moves into packages/llm once a second gateway consumer exists #325

Description

@ckrough

Follow-up from #93 (shared gateway client package), promised in PR #321. Deliberately deferred, not forgotten: it needs a second consumer to exist before the right shape is knowable.

#93 moved build_gateway_client into packages/llm but left the settings side in each service. retriever.config.Settings still owns:

  • llm_gateway_base_url (the computed field resolving LLM_GATEWAY_URL, else CLOUDFLARE_ACCOUNT_ID + CLOUDFLARE_GATEWAY_ID, else ValueError)
  • gateway_token_for(scope) (scoped-token lookup with fallback to the shared llm_gateway_token)
  • the gateway fields themselves (llm_gateway_token, llm_gateway_auth_header, the per-scope tokens)

GatewaySettings in packages/llm is a structural typing.Protocol describing that surface, not an implementation of it. So the second service to call the gateway must reimplement all three, which is the copy-paste #93 set out to prevent, one layer up.

This was the right call for #93: with one consumer, any shared base class would be designed against a sample size of one, and a pydantic settings mixin would have coupled the package to pydantic-settings for no present benefit. The question is genuinely open until a second service has real requirements.

Do this when a second service (biowriter, per ADR 0028, or petdata if it starts making model calls) lands its first gateway call. Resolve then:

  • Whether the shared piece is a pydantic-settings mixin, a plain resolver function taking the env values, or documentation plus the Protocol as-is.
  • Whether the base-URL resolution (LLM_GATEWAY_URL / Cloudflare account+gateway derivation) moves into packages/llm, given Validate gateway base URL scheme and host in build_gateway_client #323 will add validation there anyway.
  • Whether gateway_token_for's scoped-token-with-fallback logic moves, and if so how per-service env prefixes (PETDATA_, etc.) are handled.
  • Whether retriever migrates onto the shared piece in the same PR or a follow-up.

Acceptance Criteria

  • A decision is recorded, in packages/llm/README.md or an ADR amendment, even if the decision is to keep the Protocol-only split.
  • If shared code is added, no service reimplements base-URL resolution or scoped-token fallback.
  • packages/llm still imports nothing from any service (ADR 0001).
  • Retriever's gateway behavior is unchanged, verified by its existing scoped-token tests.

Metadata

Metadata

Assignees

No one assigned

    Labels

    module:platformCross-cutting platform / foundationtaskTests, docs, refactoring, non-feature work

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions