Skip to content

Add multi-LLM clients, quota tracking and quota-board UI#5

Merged
Lzxpan merged 2 commits into
masterfrom
codex/analyze-proposed-modifications-and-features-px9k51
Mar 14, 2026
Merged

Add multi-LLM clients, quota tracking and quota-board UI#5
Lzxpan merged 2 commits into
masterfrom
codex/analyze-proposed-modifications-and-features-px9k51

Conversation

@Lzxpan

@Lzxpan Lzxpan commented Mar 14, 2026

Copy link
Copy Markdown
Owner

Motivation

  • Add support for multiple LLM backends (local vLLM Pixtral, Mistral Pixtral, Groq Llama4 and Gemini) and allow automatic fallback between configured credentials.
  • Track and persist daily usage and simple rate limits so users can view per-engine consumption and identify quota/rate-limit failures.

Description

  • Introduce LlmClients.cs with LlmProvider, LlmCredential, OpenAiCompatibleClient, and LlmErrorPolicy to call OpenAI-compatible endpoints and unify text/image requests.
  • Add DailyQuotaTracker.cs to persist daily usage to usage_daily.json, provide defaults for common models, and expose snapshot/recording APIs.
  • Extend the main form (Form1.cs and Form1.Designer.cs) to add API key/model fields for Pixtral, Mistral Pixtral and Llama4, implement BuildLlmCredentials(), fallback methods TranslateTextWithFallbackAsync() and TranslateImageWithFallbackAsync(), and wire quota recording and saving.
  • Add a simple QuotaBoardForm embedded in Form1.cs to visualize today's per-engine usage and a toolbar button btnQuotaBoard to open it.
  • Persist new settings by adding properties in Properties/Settings.* and update App.config defaults for the new keys.
  • Update project file to include the new source files (LlmClients.cs, DailyQuotaTracker.cs) and include UI/behaviour changes like a close-confirmation dialog.

Testing

  • No automated tests were added or executed for this change.

Codex Task

@Lzxpan Lzxpan merged commit b15c0df into master Mar 14, 2026
0 of 2 checks passed
@chatgpt-codex-connector

Copy link
Copy Markdown

💡 Codex Review

if (cmbModel_Pixtral.SelectedItem != null)
{

P1 Badge Skip local Pixtral credential unless explicitly configured

BuildLlmCredentials() unconditionally adds the local vLLM Pixtral endpoint whenever cmbModel_Pixtral has a selected item, and Form1_Load always selects one by default. That means users who only configured cloud keys still get http://127.0.0.1:8000/v1 inserted into the fallback chain; when that local server is absent, Translate*WithFallbackAsync() returns this non-quota failure immediately and never reaches later configured providers. Gate this credential behind explicit user intent/configuration so fallback can reach actually configured backends.


catch (Exception ex)
{
return new GeminiResult
{
HttpStatus = 0,

P1 Badge Let transport errors trigger fallback to the next credential

OpenAiCompatibleClient.PostChatCompletion() catches network/transport exceptions and returns a normal GeminiResult (HttpStatus = 0) instead of throwing. In the fallback methods, retry-to-next-provider on network problems is only implemented in the exception path (catch + IsRetryableNetworkError), so these wrapped failures are treated as terminal errors and stop fallback. This breaks multi-provider resilience during transient connection issues; either rethrow retryable exceptions here or classify HttpStatus == 0 as retryable in fallback.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant