Request - support local/arbitrary inference endpoints by adding base_url parameter

[platform-kit]

Excellent work! This is something I've been hoping to see for some time.

Please consider decoupling from OpenAI. It should be trivial to add support for other models since the OpenAI API spec has been implemented almost universally across various inference providers, local tools (like Ollama), etc.

The minimum change would be to allow the user to specify a base_url variable, instead of or in addition to the API key, then pass that to any instances where the OpenAI SDK is used to call the inference endpoint.

[johnblommers]

Let me add my voice to request support for local-only inference, perhaps using Ollama so we can pick and choose our own LLM and run them on our own hardware.

[claws61821]

I echo this request, although I might recommend Llama.CPP instead of Ollama simply because it's the upstream fork and has broader presence in third-party projects

[Pythonpa]

Yes, this also counts as a plea from me. I hope that support for other third-party APIs or local models can be added. I would be very grateful.