feat: extract shared gateway client into packages/llm (#93)

docs/adr/0034-packages-llm-gateway-client.md

@@ -0,0 +1,27 @@
+# ADR 0034: Shared LLM gateway client at `packages/llm/`
+
+- Status: accepted
+- Date: 2026-07-27
+- Deciders: project owner
+- Relates to: ADR 0022 (monorepo structure), ADR 0028 (llm-gateway-consolidation), ADR 0031 (shared-design-system-package)
+
+## Context
+
+ADR 0028 consolidated all outbound model traffic (chat, embeddings, moderation) onto one OpenAI-compatible gateway, authenticated with a single gateway token, built through one `build_gateway_client` helper. Its follow-ups said to promote that constructor into a shared package once one existed, so petdata and biowriter would import it rather than copy it. `packages/` now exists (ADR 0031 created it for the design system), so the promotion can happen.
+
+`build_gateway_client` lived in `services/retriever/src/retriever/infrastructure/llm/gateway_client.py`, reading retriever's `Settings` object directly. Shared code cannot import a service's config: ADR 0022 puts shared code in `packages/`, imported directly by the services, and a package that imported retriever's `Settings` would recreate that coupling one layer up instead of removing it.
+
+## Decision
+
+`build_gateway_client` moves out of retriever into a new shared package, `packages/llm/` (distribution name `evermore-llm`, import name `evermore_llm`). The service-local `gateway_client.py` is deleted. Retriever consumes the package via `[tool.uv.sources] evermore-llm = { path = "../../packages/llm", editable = true }`, the same mechanism it already uses for `packages/auth`.
+
+- **Settings decoupling.** Because the package cannot import a service's config, it defines a structural `typing.Protocol` named `GatewayConfig`, naming the four members the constructor reads: `llm_gateway_auth_header`, `llm_gateway_token`, the read-only property `llm_gateway_base_url`, and `gateway_token_for`. Retriever's `Settings` satisfies `GatewayConfig` structurally with no change to `Settings` itself, and mypy checks the match at each call site. `packages/auth` uses the same service-agnostic pattern, so `packages/llm` follows precedent rather than inventing a new one.
+- **Scope boundary.** `packages/llm` owns shared model-call infrastructure: the gateway client constructor, the `GatewayConfig` protocol, and the `GatewayScope` Literal. Service-specific wiring, provider selection, fallback chains, retry and circuit-breaker policy, and dependency-injection wiring, stays in the service that needs it. Cross-service domain contracts go in `packages/schema`, not here. Telemetry for outbound model calls is out of scope and belongs to `packages/observability` (#114).
+- **Consumers, and the deferral.** retriever is the one consumer today. petdata makes no gateway calls and carries no `openai` dependency, so adding `packages/llm` there now would add an unused dependency; petdata imports the package when it starts making gateway calls. biowriter is not yet scaffolded as a Python service, so its wiring is deferred to #64. Both services import the same shared helper when their gateway needs arrive; the scope boundary above is what stops a third copy of the constructor from appearing before then.
+
+## Consequences
+
+- retriever's gateway wiring now imports `evermore_llm.build_gateway_client` instead of a service-local module; the tests for the constructor moved to `packages/llm/tests/` with the code they test.
+- `packages/llm` has no standalone CI job. It is verified transitively through the retriever CI job: the `packages/**` paths filter in `.github/workflows/ci.yml` already triggers that job on changes here, and retriever's test suite exercises the shared client against its own `Settings`. This is the same coverage arrangement `packages/auth` and `packages/schema` run under today.
+- Adding petdata or biowriter as a second consumer becomes an editable path dependency plus satisfying `GatewayConfig`, not a copy of the constructor.
+- A future contributor can tell from placement alone whether new gateway code belongs in `packages/llm` (shared across services) or in a service's own infrastructure layer (specific to one service).

docs/adr/README.md

@@ -79,3 +79,4 @@ One line per record: number, slug, status. Where a record is superseded, the suc
 - 0031 shared-design-system-package: accepted
 - 0032 supabase-auth-cookie-non-httponly: accepted
 - 0033 listing-sync-verification: proposed
+- 0034 packages-llm-gateway-client: accepted

packages/llm/README.md

@@ -0,0 +1,60 @@
+# evermore-llm
+
+Shared OpenAI-compatible LLM gateway client builder for the Evermore
+services. One canonical copy of the gateway-transport plumbing, consumed by
+every service that makes outbound model calls (chat, embeddings, moderation):
+
+- `evermore_llm.build_gateway_client` constructs an `AsyncOpenAI` client
+  pointed at the gateway's OpenAI-compatible endpoint, given a config object
+  and an optional traffic-class scope.
+- `evermore_llm.GatewayConfig` is the structural `typing.Protocol` naming the
+  four members the builder reads (`llm_gateway_auth_header`,
+  `llm_gateway_token`, `llm_gateway_base_url`, `gateway_token_for`). A
+  service's own settings class satisfies it structurally, with no import of
+  this package's types required at the settings definition site.
+- `evermore_llm.GatewayScope` is the `Literal["chat", "embeddings",
+  "moderation"]` traffic-class type shared by the config protocol and every
+  consumer.
+
+The gateway-consolidation decision and the reasoning behind the
+`GatewayConfig` seam are owned by
+[`docs/adr/0028-llm-gateway-consolidation.md`](../../docs/adr/0028-llm-gateway-consolidation.md)
+and
+[`docs/adr/0034-packages-llm-gateway-client.md`](../../docs/adr/0034-packages-llm-gateway-client.md);
+this README covers only the package's own API surface. Shared model-call
+infrastructure lives here; service-specific wiring (which settings class,
+which FastAPI dependency graph builds the client) and domain contracts
+(`Package`, `Composition`, and the rest of the data spine) do not, those
+belong to `packages/schema`.
+
+## Usage
+
+```python
+from evermore_llm import build_gateway_client
+
+client = build_gateway_client(settings, scope="chat")
+```
+
+`settings` is any object that structurally satisfies `GatewayConfig`; a
+service's existing settings class needs no changes to qualify.
+
+## Deviations from repo defaults
+
+Pinned to Python 3.13+, not the repo's 3.14 floor: it matches its current
+consumer, `services/retriever`. ADR
+[`0024-standardized-tech-stack.md`](../../docs/adr/0024-standardized-tech-stack.md)
+already tracks retriever's move to 3.14 as outstanding work, not
+grandfathered.
+
+## Security contract
+
+`gateway_token_for` is part of this package's security contract, not just its type
+signature. An implementation must return the token scoped to the requested traffic class
+and fall back to the shared token only when the scoped one is unset. Returning the shared
+token for every scope type-checks against `GatewayConfig` but defeats the blast-radius
+narrowing that scoped tokens exist to provide (see issue #228).
+
+`build_gateway_client` sends no gateway auth header at all when the resolved token is
+empty, and passes the placeholder `api_key="unused"`. A consumer that leaves its token
+unset therefore gets a client that looks working but authenticates nothing. Set the
+gateway token in every environment that reaches a real gateway.

packages/llm/pyproject.toml

@@ -0,0 +1,85 @@
+[project]
+name = "evermore-llm"
+version = "0.1.0"
+description = "Shared OpenAI-compatible LLM gateway client builder for Evermore"
+readme = "README.md"
+license = "Apache-2.0"
+requires-python = ">=3.13"
+dependencies = [
+    "openai>=1.60",
+    "pydantic>=2.9",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=8.0",
+    "mypy>=1.13",
+    "ruff>=0.8",
+]
+
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/evermore_llm"]
+
+[tool.ruff]
+target-version = "py313"
+line-length = 88
+src = ["src", "tests"]
+
+[tool.ruff.lint]
+select = [
+    "E",      # pycodestyle errors
+    "W",      # pycodestyle warnings
+    "F",      # Pyflakes
+    "I",      # isort
+    "B",      # flake8-bugbear
+    "C4",     # flake8-comprehensions
+    "UP",     # pyupgrade
+    "ARG",    # flake8-unused-arguments
+    "SIM",    # flake8-simplify
+    "TCH",    # flake8-type-checking
+    "PTH",    # flake8-use-pathlib
+    "ERA",    # eradicate
+    "PL",     # Pylint
+    "RUF",    # Ruff-specific
+]
+ignore = [
+    "PLR0913",  # Too many arguments
+    "PLR2004",  # Magic value comparison
+]
+
+[tool.ruff.lint.isort]
+known-first-party = ["evermore_llm"]
+
+[tool.mypy]
+python_version = "3.13"
+strict = true
+warn_return_any = true
+warn_unused_configs = true
+disallow_untyped_defs = true
+disallow_incomplete_defs = true
+check_untyped_defs = true
+disallow_untyped_decorators = true
+no_implicit_optional = true
+warn_redundant_casts = true
+warn_unused_ignores = true
+warn_no_return = true
+follow_imports = "normal"
+show_error_codes = true
+
+[[tool.mypy.overrides]]
+module = "tests.*"
+disallow_untyped_defs = false
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+pythonpath = ["src", "."]
+addopts = [
+    "-ra",
+    "-q",
+    "--strict-markers",
+    "--strict-config",
+]

packages/llm/src/evermore_llm/__init__.py

@@ -0,0 +1,18 @@
+# Copyright (C) 2025 Backchain LLC
+# SPDX-License-Identifier: Apache-2.0
+
+"""Shared OpenAI-compatible LLM gateway client builder for Evermore.
+
+This package is the single canonical source for the AsyncOpenAI client
+builder (:func:`build_gateway_client`) that every service routes outbound
+model calls (chat, embeddings, moderation) through, decoupled from any one
+service's settings via the structural :class:`GatewayConfig` protocol.
+"""
+
+from evermore_llm.gateway_client import (
+    GatewayConfig,
+    GatewayScope,
+    build_gateway_client,
+)
+
+__all__ = ["GatewayConfig", "GatewayScope", "build_gateway_client"]

services/retriever/src/retriever/infrastructure/llm/gateway_client.py → packages/llm/src/evermore_llm/gateway_client.py

@@ -9,21 +9,38 @@
 so this builder carries no gateway-specific identifiers. An optional ``scope``
 narrows the token used to one traffic class (chat, embeddings, moderation),
 shrinking the blast radius of a single leaked token; see
-``Settings.gateway_token_for``.
+``GatewayConfig.gateway_token_for``.
 """
 
 from __future__ import annotations
 
-from typing import TYPE_CHECKING
+from typing import TYPE_CHECKING, Literal, Protocol
 
 from openai import AsyncOpenAI
 
 if TYPE_CHECKING:
-    from retriever.config import GatewayScope, Settings
+    from pydantic import SecretStr
+
+# Per-traffic-class gateway token scope. Narrows the blast radius of a
+# leaked token to one model traffic class (chat, embeddings, moderation)
+# instead of all gateway traffic authenticated by the shared token.
+GatewayScope = Literal["chat", "embeddings", "moderation"]
+
+
+class GatewayConfig(Protocol):
+    """Structural config surface that ``build_gateway_client`` reads from."""
+
+    llm_gateway_auth_header: str
+    llm_gateway_token: SecretStr
+
+    @property
+    def llm_gateway_base_url(self) -> str: ...
+
+    def gateway_token_for(self, scope: GatewayScope | None) -> SecretStr: ...
 
 
 def build_gateway_client(
-    settings: Settings,
+    config: GatewayConfig,
     *,
     scope: GatewayScope | None = None,
     timeout_seconds: float = 30.0,
@@ -37,34 +54,34 @@ def build_gateway_client(
     is ignored in favor of the gateway's stored keys plus the auth header.
 
     Args:
-        settings: Application settings supplying the gateway base URL, token,
-            and auth header name.
+        config: Configuration supplying the gateway base URL, token, and auth
+            header name.
         scope: Optional traffic-class scope ("chat", "embeddings",
             "moderation"). When given, the token is resolved via
-            ``settings.gateway_token_for(scope)``, which prefers the matching
+            ``config.gateway_token_for(scope)``, which prefers the matching
             scoped token and falls back to the shared ``llm_gateway_token``
             when the scoped token is unset. When omitted (None), the shared
             ``llm_gateway_token`` is used directly.
         timeout_seconds: Request timeout in seconds.
 
     Returns:
-        An AsyncOpenAI client whose base URL is settings.llm_gateway_base_url
+        An AsyncOpenAI client whose base URL is config.llm_gateway_base_url
         and which sends the configured auth header when a gateway token is set.
 
     Raises:
         ValueError: If no LLM gateway is configured (propagated from
-            settings.llm_gateway_base_url).
+            config.llm_gateway_base_url).
     """
     if scope is None:
-        token = settings.llm_gateway_token.get_secret_value()
+        token = config.llm_gateway_token.get_secret_value()
     else:
-        token = settings.gateway_token_for(scope).get_secret_value()
+        token = config.gateway_token_for(scope).get_secret_value()
     default_headers = (
-        {settings.llm_gateway_auth_header: f"Bearer {token}"} if token else None
+        {config.llm_gateway_auth_header: f"Bearer {token}"} if token else None
     )
     return AsyncOpenAI(
         api_key=token or "unused",
-        base_url=settings.llm_gateway_base_url,
+        base_url=config.llm_gateway_base_url,
         timeout=timeout_seconds,
         default_headers=default_headers,
     )