Skip to content

fix(auth): auto-switch profile on genuine exhaustion (regression from #374's false-cascade fix)#391

Open
andrei-hasna wants to merge 1 commit into
mainfrom
fix/auth-profile-autoswitch-on-real-exhaustion
Open

fix(auth): auto-switch profile on genuine exhaustion (regression from #374's false-cascade fix)#391
andrei-hasna wants to merge 1 commit into
mainfrom
fix/auth-profile-autoswitch-on-real-exhaustion

Conversation

@andrei-hasna

Copy link
Copy Markdown
Contributor

Summary

The /config "Auth profile auto-switch" toggle no longer switches profiles when the current profile is genuinely exhausted (real usage-limit / 429, e.g. "You've hit your usage limit, try again Jul 28"). The turn just parks on a self-heal retry until the (days-away) reset. This restores the legitimate switch without reviving the cross-agent false-cascade that #374 fixed.

Root cause (regression from #374)

#374 (a31719336) stopped a false weekly-limit cascade by making rolling account/rateLimits/updated snapshots display-only and gating all exhaustion bookkeeping + auto-switch behind the account-verified RateLimitSnapshotSource::AccountUsage read (codex-rs/tui/src/chatwidget/rate_limits.rs:314-339).

That over-corrected. Before #374 the rolling snapshot was also the trigger that drove auto-switch on genuine mid-turn exhaustion: it populated auth_profile_auto_switch_snapshots_by_limit_id and called maybe_auto_switch_auth_profile_for_rate_limit. After #374, on genuine exhaustion:

  • The switch map (auth_profile_auto_switch_snapshots_by_limit_id) is empty — the limit is crossed mid-turn, before the ~60s authoritative heartbeat re-reads it, and the heartbeat is then suppressed for the exhausted profile (auth_profile_popups.rs:585-591).
  • Both genuine-exhaustion fallbacks read that empty map and bail: try_auth_profile_switch_after_reset_unavailable (usage_limit_reset/automatic.rs:225-240) and maybe_auto_switch_auth_profile_before_user_turn (rate_limits.rs:409).
  • The usage-limit turn-error path only switched when manual_usage_limit_reset_is_active() was true (turn_runtime.rs on_rate_limit_error), so with auto-reset unavailable it just scheduled a self-heal retry for the reset instant — "it just stops."

#374's own PR body flagged the follow-ups this closes ("consider verifying target capacity before switching"; choose_profile_for_auto_switch picking Unknown blindly).

Fix

Drive auto-switch from the current profile's own, authoritative usage-limit / 429 turn failure, which is authoritative for the current profile (the app server rejected this profile's own turn) and cannot be produced by a sibling agent's identity-less rolling snapshot:

  • New try_auth_profile_switch_for_usage_limit (rate_limits.rs): prefers a cached account-verified exhausted window; when none is cached, synthesizes the trigger window from the operator's enabled auto-switch config (weekly preferred, else 5h; None when both opted out so a disabled window is never switched on) and reuses the existing candidate selection (cycles to an Unknown-but-untried profile; bails — no loop — when every alternate is exhausted). The reset instant is parsed from the error text so repeated genuine failures get distinct trigger keys and can cycle across profiles.
  • turn_runtime.rs on_rate_limit_error: the reset-unavailable branch now calls it (gated to genuine UsageLimit, not transient 429/overload), dropping the too-narrow manual_usage_limit_reset_is_active() requirement.
  • usage_limit_reset/automatic.rs fallback_auth_profile_switch_after_reset_unavailable: also routes through it, so the auto-reset path recovers even when the authoritative read hasn't cached the exhausted window yet.

Preserves #374's no-false-cascade invariant

Rolling snapshots stay strictly display-only and never seed the switch map. The new switch is reachable only from on_rate_limit_error / the post-error reset fallback — never from on_rolling_rate_limit_snapshot — so a sibling agent's foreign snapshot still cannot trigger a switch.

Tests (tui/src/chatwidget/tests/status_and_layout.rs)

  • genuine_usage_limit_error_auto_switches_without_a_cached_snapshot — the regression: exhausted current profile + auto-switch on + another profile configured + empty map -> switches.
  • rolling_snapshot_alone_never_switches_and_never_seeds_the_switch_mapfix(auth): stop false-positive weekly-limit cascade across auth profiles #374 preserved: rolling 100% snapshot never switches and never seeds the switch map.
  • genuine_usage_limit_error_does_not_loop_when_all_profiles_exhausted — graceful: all alternates exhausted -> no switch, no loop (twice).

Verification (Blacksmith, 16 vCPU)

  • Full codex-tui nextest suite: 3478 passed, 4 skipped, 0 failed.
  • 3 new tests: PASS.
  • cargo fmt --all -- --config imports_granularity=Item --check: clean (exit 0).
  • cargo clippy -p codex-tui --tests --all-targets -- -D warnings: clean (exit 0; only a pre-existing transitive-dep proc-macro-error2 future-incompat note, unrelated).

Do not merge/tag yet — hand-off for 0.1.78.

…374's false-cascade fix)

#374 stopped a false weekly-limit cascade by making rolling
account/rateLimits/updated snapshots display-only and gating all exhaustion
bookkeeping + auto-switch to the account-verified AccountUsage read. That over-
corrected: the rolling snapshot was also what drove auto-switch on GENUINE
mid-turn exhaustion (it populated auth_profile_auto_switch_snapshots_by_limit_id
and called maybe_auto_switch_auth_profile_for_rate_limit). After #374, when the
current profile is really exhausted the switch map is empty (the ~60s
authoritative heartbeat has not re-read yet and is then suppressed), and the
usage-limit turn-error path only switched when a manual reset was active — so
with auto-reset unavailable the failed turn just parked on a self-heal retry
until the (days-away) reset instead of switching.

Fix: drive auto-switch from the current profile's OWN authoritative usage-limit
/ 429 turn error (turn_runtime.rs on_rate_limit_error and the reset-unavailable
fallback in usage_limit_reset/automatic.rs). This is authoritative for the
current profile and immune to the #374 misattribution: a sibling agent's
identity-less rolling snapshot never fails THIS profile's own turn. It prefers a
cached account-verified exhausted window and otherwise synthesizes the trigger
window from the operator's enabled auto-switch config (weekly preferred, else
5h), reusing the existing candidate selection that cycles to an Unknown-but-
untried profile and bails (no loop) when every alternate is exhausted.

Rolling snapshots remain strictly display-only and never seed the switch map, so
#374's no-false-cascade invariant holds.

Tests (tui/src/chatwidget/tests/status_and_layout.rs):
- genuine_usage_limit_error_auto_switches_without_a_cached_snapshot (the regression)
- rolling_snapshot_alone_never_switches_and_never_seeds_the_switch_map (#374 preserved)
- genuine_usage_limit_error_does_not_loop_when_all_profiles_exhausted (graceful)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant