Skip to content

fix: test embedding models through embeddings endpoint - #97

Draft
z125316840-code wants to merge 1 commit into
frappe:developfrom
z125316840-code:agent/fix-embedding-model-connection-test
Draft

fix: test embedding models through embeddings endpoint#97
z125316840-code wants to merge 1 commit into
frappe:developfrom
z125316840-code:agent/fix-embedding-model-connection-test

Conversation

@z125316840-code

Copy link
Copy Markdown

Summary

  • detect embedding models using LiteLLM metadata
  • fall back to the model name for custom OpenAI-compatible embedding models missing from LiteLLM's registry
  • test embedding models with litellm.embedding() and explicit encoding_format="float"
  • keep chat models on litellm.completion()
  • add regression coverage for mapped/unmapped embeddings and chat models

Root cause

FlowModel.test_connection() always sent a chat completion request. Embedding-only models therefore hit /chat/completions and returned a misleading 404 even when their model, credentials, and provider routing were valid. With the OpenAI-compatible provider used for live verification, omitting encoding_format also caused LiteLLM 1.83.7 to send an empty value rejected by the provider; using the standard float format resolves that compatibility issue.

Verification

  • git diff --check
  • Python 3.14 bytecode compilation in the deployment image
  • 25 Flow Model integration tests passed
  • live deployment verified against a custom OpenAI-compatible text-embedding-v4 model: Connection OK
  • chat-model regression verifies litellm.completion() remains unchanged

Closes #96

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test_connection on Flow Model always fails for embedding models (chat ping via litellm.completion)

1 participant