Skip to content

Add LLM fallback clients, daily quota tracker and quota-board UI#6

Merged
Lzxpan merged 1 commit into
masterfrom
codex/analyze-proposed-modifications-and-features-5v2cr2
Mar 14, 2026
Merged

Add LLM fallback clients, daily quota tracker and quota-board UI#6
Lzxpan merged 1 commit into
masterfrom
codex/analyze-proposed-modifications-and-features-5v2cr2

Conversation

@Lzxpan

@Lzxpan Lzxpan commented Mar 14, 2026

Copy link
Copy Markdown
Owner

Motivation

  • Support multiple LLM backends (local vLLM, Mistral, Groq) and automatically failover between configured API keys when a provider is rate/quotas limited.
  • Provide per-model daily usage accounting so users can see and persist daily quotas and errors to avoid unexpected rate limits.
  • Expose new model/API key settings and a simple UI to view today’s engine usage without modifying existing workflows.

Description

  • Added a daily usage tracking subsystem in DailyQuotaTracker.cs that records per-provider/model request counts, token usage, last error, and default daily/RPM limits and persists to usage_daily.json via DailyQuotaTracker.LoadOrCreate and Save.
  • Implemented OpenAI-compatible client and multi-provider credential model in LlmClients.cs with OpenAiCompatibleClient, LlmCredential, LlmProvider, and error policy helpers; added TranslateTextWithFallbackAsync and TranslateImageWithFallbackAsync in Form1.cs to try configured credentials sequentially and record results to the quota tracker.
  • Extended UI and settings: added input fields and settings entries for ApiKey_Pixtral, ModelName_Pixtral, ApiKey_MistralPixtral, ModelName_MistralPixtral, ApiKey_Llama4, and ModelName_Llama4 in Form1.Designer.cs, Properties/Settings.*, and App.config defaults, and wired them into Form1 load/save logic.
  • Added a lightweight QuotaBoardForm inside Form1.cs and a btnQuotaBoard button to show the current day’s usage snapshot returned by DailyQuotaTracker.GetSnapshot, and added logic to persist usage on exit and on each recorded request.
  • Minor UX change: Form1_FormClosing now prompts the user with choices to exit or minimize to tray before saving settings and usage.

Testing

  • No automated tests were added or executed as part of this change (no CI test suite changes).

Codex Task

@Lzxpan Lzxpan merged commit b2731a3 into master Mar 14, 2026
2 checks passed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f567fd966e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Form1.cs
Comment on lines +877 to +881
if (cmbModel_Pixtral.SelectedItem != null)
{
list.Add(new LlmCredential
{
Provider = LlmProvider.Pixtral12BLocalVllm,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Skip implicit local vLLM credential when not configured

This block always adds the local Pixtral credential whenever a model is selected, and Form1_Load always selects one by default, so users who only configured Mistral/Groq still hit http://127.0.0.1:8000/v1 first. If local vLLM is not running, the first request fails with a non-quota error and TranslateTextWithFallbackAsync/TranslateImageWithFallbackAsync return immediately instead of reaching later configured providers, causing translation to fail despite valid remote API keys.

Useful? React with 👍 / 👎.

Comment thread Form1.cs

InitializeTrayIcon();

_quotaTracker = DailyQuotaTracker.LoadOrCreate(Path.Combine(Application.StartupPath, "usage_daily.json"));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Persist quota data in a user-writable directory

The quota tracker file is created under Application.StartupPath, which is commonly read-only for normal users when the app is installed under Program Files. Because DailyQuotaTracker.Save() suppresses IO exceptions, quota history silently fails to persist in that environment. Using a per-user data path (for example %LocalAppData%) would avoid silent data loss.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant