Add tool use MVP to MLX models

[thomasantony]

This is an MVP for tool usage support for MLX-based models. It uses apply_chat_template to add tool definitions to the chat template (assuming the model already has a tool call template). The plugin tries to detect the tool format. And if it cannot identify it, it adds a generic tool calling prompt of its own to the system prompt (this is admittedly a bit janky and can be removed if deemed too unpolished).

Tool parsing is done based on the detected tool format and a set of fixed rules.
Tool results are added to the conversation using the template.

It also works with thinking models like qwen3. Multiple tool calls in a single request is supported.

This has been tested so far with the following models:

• mxmcc/Llama-xLAM-2-8b-fc-r-mlx-8Bit
• mlx-community/Llama-3.2-3B-Instruct-4bit
• mlx-community/Qwen3-8B-4bit
• mlx-community/gemma-3-12b-it-4bit-DWQ

Some models struggle if asked for more than one tool call at once. I have had good results with the Qwen3-8b model listed above.

[simonw]

I'd rather not include the custom tools system prompt stuff just because of the added complexity and future maintenance burden.

I'm really interested in exploring that kind of thing separate though. I'd love to be able to offer prompt-driven tool support in LLM core that works with any model (from any plugin) that doesn't have its own tool support.

I haven't figured out what shape that would take yet.

[thomasantony]

I have removed the commit with the injected template and fixed a couple of bugs.

[thomasantony]

Hey @simonw, did you get a chance to review this?