Skip to content

Provider support is combinatorial: eight agent files × N providers, each wired by hand #88

Description

@alpibrupa

Every agent file — build, plan, explore, refactor, spec_agent, test_agent, review, bar — carries one function per provider: anthropic_agent, openai_agent, ollama_agent, litellm_agent, vllm_agent, mistral_agent, and so on. session.lex then has a match over provider tag × mode that dispatches to the right one. Adding a provider means touching nine files. Adding a mode means touching every provider's row.

That shape produced a bug this week, and it will produce more. ollama_local() hardcoded localhost:11434 while litellm() and vllm_local() read theirs from the environment — because each was written separately, on a different day, and nothing made them agree. lex-llm#49 fixed that one. The structure that let it happen is untouched.

What lex-loom does instead

lex-loom has no per-provider anything. Everything goes through one OpenAI-compatible adapter pointed at LiteLLM; a provider is a row in litellm/config.yaml:

- model_name: qwen3.8:27b-mlx
  litellm_params:
    model: ollama_chat/qwen3.8:27b-mlx
    api_base: os.environ/OLLAMA_BASE_URL

Two environment variables (LITELLM_BASE_URL, MODEL) select everything. Provider quirks live in config, where the LiteLLM project maintains them, rather than in Lex, where this project does. Its config already carries things lex-code has nowhere to put — max_tokens floors for thinking models, merge_reasoning_content_in_choices, the tool-count warning.

lex-code already has litellm_agent in every file. It's the same shape as the others rather than the replacement for them.

What this would look like here

Not a rewrite. Three steps, each independently useful:

  1. One agent constructor per mode, taking (provider :: prov.Provider, model :: prov.ModelRef, tools :: List[t.Tool], max_steps :: Int). The eight *_agent() functions per file become eight one-line calls, or disappear into a table. session.lex's match collapses to a lookup.
  2. Provider selection in one place — a providers.resolve(tag) that returns (Provider, ModelRef, tool policy), reading env once. The ollama_local bug becomes structurally impossible because there is one function that reads base URLs.
  3. Then decide whether the native adapters (Ollama, vLLM, Mistral) earn their keep or whether litellm becomes the default and they become config rows. That is a real design question, and the eval harness in An eval harness: nothing measures whether lex-code writes good Lex #86 is how to answer it: run the same tasks via ollama native and via litellm → ollama_chat and see if the scores differ. If they don't, the native adapter is maintenance with no return.

Step 3 should not be decided before steps 1–2 make it cheap to try, and not before #86 makes it measurable.

Why not just leave it

Because the next provider bug will look exactly like the last one: a difference between two hand-written functions that were supposed to be the same, found by someone who ran it, months after it shipped. #83 was twenty of those in the tool layer. This is the same pattern one layer up.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions