Skip to content

The local-model toolset is sized by guess, not measurement #87

Description

@alpibrupa

The Ollama build agent offers nine tools via minimal_tools(). The other providers get all_tools(): forty-four, nineteen of them the vcs family.

lex-loom's LiteLLM config carries this note from running local models for real:

Tool calling with these models is unreliable under large context (10+ tools). Prefer qwen3-coder:30b or devstral-small-2 for code agent tasks.

Nine is under that line by one. Nothing in this repo says why nine, or which nine, or whether five would score the same. bar_minimal_tools() already exists as a second hand-picked cut for a different mode, which is the pattern starting to spread: per-mode lists chosen by feel.

What's actually in the forty-four

Most of the surface is a thin wrapper over a lex subcommand the model can already reach through bash. The nineteen vcs tools are the clearest case — every one is proc.run("lex", ["branch" | "op" | "merge", ...]) plus argument marshalling. They exist so the model has a typed way in, which is a real benefit for a large model that reads schemas well, and a real cost for a small one that has to hold nineteen schemas in a context it doesn't have.

There's also lex_cli_tools(), an escape hatch that runs an arbitrary lex line and is registered in no mode's default set. That's the honest floor: with bash and lex_cli, the model can do everything the other forty-two do. The forty-two are a UX layer, and a UX layer should be sized to the user.

What to do

This is the first question the eval harness in #86 should answer, and it should not be decided before then:

  1. Run the golden tasks against the Ollama path with the current nine tools.
  2. Run again with the floor: read, edit, write, grep, glob, bash, lex_check. Seven.
  3. Run again with five: drop write (edit can create) and glob (bash can).
  4. Keep the smallest set whose score matches the largest.

Then apply the same procedure to the other modes' local-model variants rather than hand-picking them. The number that comes out is the number; the point of the exercise is to stop having an opinion about it.

A cheaper thing to do first

The per-tool descriptions are what the model actually reads, and several are long. lex_audit's is three lines. For a 4K-context model the descriptions alone are a meaningful fraction of the budget. Measuring the serialised size of minimal_tools()'s schemas is a five-minute check that may say more than the count does.

Depends on #86 for the measurement; the size audit needs nothing.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions