fix(config): survive a rate-limited embedding probe when saving config#69
Merged
Merged
Conversation
jaymeklein
force-pushed
the
fix/config-save-survives-provider-429
branch
2 times, most recently
from
July 16, 2026 18:41
75959ae to
c49a159
Compare
…nfig Saving the embedding config validates it by building the adapters, which sizes the pgvector store from the embedding vector dimension. For Gemini that dimension is learned by probing the provider (GeminiAdapter.dimensions embeds a short string). When the provider returns HTTP 429 the probe raised and the save was rejected - first as a 422 in the API dry run, then as a 409 when each worker rebuilt its own adapters on the reload and hit the same 429. So editing max requests/min, the very knob meant to relieve 429s, was impossible while the provider was throttled. A 429 means the provider is reachable and the key is valid; it is throttled, not misconfigured, and must never block or roll back a config save. Add a shared resolve_dimensions() helper (api/adapters/embeddings) used by BOTH the API apply and the worker reload: it probes the provider first - so a bad key / unreachable host still fails fast, and a stale size self-heals once the provider answers - but on a RateLimitError, when the embedding model shape is unchanged (provider, model, base_url, output dimensionality), it reuses the existing store's known dimension instead of failing. Expose PgvectorAdapter.dimensions for that reuse. The API always has a boot-built store to reuse, so an unchanged-model save goes straight through; only a model change it genuinely can't size while throttled still returns a transient 503. A worker may have no store yet (a process throttled since its last restart never built one), so it can't always reuse: there it *defers* - adopts the new embedder so the change takes effect, and nulls the store for a lazy rebuild on next use - rather than error-acking and forcing a rollback. Any non-429 failure still propagates, so a genuinely bad config rolls back. Tests: a focused suite for resolve_dimensions / same_embedding_shape; the API apply survives a rate-limited probe on an unchanged model and 503s on a throttled model change; the worker reload reuses the live dimension when it has one, defers the store when it doesn't (no store built, or a model change), and still propagates non-rate-limit failures.
jaymeklein
force-pushed
the
fix/config-save-survives-provider-429
branch
from
July 16, 2026 19:00
c49a159 to
a4c0441
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Saving the embedding config rejected a valid change with
422 Invalid configurationwhenever the embedding provider was rate-limited (HTTP 429). The most perverse case:
editing max requests/min — the very throttle meant to relieve 429s — was impossible
while throttled.
Why
_build_adapterssizes the pgvector store fromembed.dimensions, and for Gemini thatproperty probes the provider live (
GeminiAdapter.dimensionsembeds a short string tolearn the vector size). A 429 during that probe bubbled into the generic
except Exception -> HTTPException(422), so a transient rate limit was reported as aninvalid configuration.
Fix
A 429 means the provider is reachable and the key is valid — it's throttled, not
misconfigured. So:
RateLimitErrorduring the dimension probe, when the embedding model shape isunchanged (
provider,model,base_url,output_dimensionality), reuse the liveadapter's already-known dimension — it cannot have changed — and let the save proceed.
transient
503("try again shortly") instead of a misleading422.Graceful degradation, matching the codebase's hard rule: a throttled provider falls back
to prior behaviour instead of turning a valid save into an error.
Tests
Two new unit tests in
tests/unit/test_config_service.py:applied, newmax_rpmpersisted);
503.Gate green: ruff + mypy clean, 707 passed / 13 skipped.