Skip to content

Explicit model instantiation should not require live model-list metadata fetch #230

@chubes4

Description

@chubes4

Summary

When a caller explicitly configures a provider and model ID, ProviderRegistry::getProviderModel() still requires a live provider model-list metadata fetch before it can instantiate the model. For OpenAI this means a configured model such as gpt-5.4 triggers GET https://api.openai.com/v1/models before the actual generation request.

This makes model metadata discovery a hard runtime dependency for explicit model usage. If the model-list request times out or is unavailable, generation fails before the configured model can be used, even though the provider's generation endpoint would validate the model ID anyway.

Observed failure

In a local WordPress Studio site running wp-ai-client through Data Machine pipeline jobs, child jobs failed with:

wp-ai-client request failed: Network error occurred while sending GET request to https://api.openai.com/v1/models: cURL error 28: Connection timed out after 30000 milliseconds

The failing generation request was not large: transcript evidence showed provider openai, model gpt-5.4, and total request size around 31 KB. Later direct checks from shell curl and WordPress PHP wp_remote_get() to /v1/models returned quickly with 401, so payload size was not the cause.

Call chain

The configured model ID still triggers GET /v1/models through this path:

  1. Caller asks the registry for an explicit provider/model pair:
    ProviderRegistry::getProviderModel( $provider, $modelId, $modelConfig )

  2. ProviderRegistry::getProviderModel() resolves the provider class and calls the provider static model factory:
    src/Providers/ProviderRegistry.php, getProviderModel(), lines 255-260 in the local copy.

  3. AbstractProvider::model() always resolves remote metadata for the requested model ID before it creates a model instance:
    src/Providers/AbstractProvider.php, model(), lines 50-58 in the local copy.

    The important line is:

    $modelMetadata = static::modelMetadataDirectory()->getModelMetadata($modelId);
  4. AbstractApiBasedModelMetadataDirectory::getModelMetadata() loads the full model metadata map:
    src/Providers/ApiBasedImplementation/AbstractApiBasedModelMetadataDirectory.php, lines 59-65.

  5. The map is populated via sendListModelsRequest() on cache miss:
    src/Providers/ApiBasedImplementation/AbstractApiBasedModelMetadataDirectory.php, lines 74-78.

    return $this->cached(self::MODELS_CACHE_KEY, fn() => $this->sendListModelsRequest(), 86400);
  6. OpenAI-compatible metadata directories implement this as a network request to models:
    src/Providers/OpenAiCompatibleImplementation/AbstractOpenAiCompatibleModelMetadataDirectory.php, lines 29-35.

    $request = $this->createRequest(HttpMethodEnum::GET(), 'models');
    $request = $this->getRequestAuthentication()->authenticateRequest($request);
    $response = $httpTransporter->send($request);
  7. The OpenAI provider resolves that path under https://api.openai.com/v1.

Expected behavior

If the caller explicitly supplies a provider and model ID, wp-ai-client should be able to instantiate a model object without a live model-list API call. Validation can happen at generation time, or the provider can synthesize conservative metadata for known model-ID families.

Why this matters

  • Explicit model selection should not require remote discovery.
  • /models metadata fetching is redundant for explicit generation requests.
  • The current behavior adds latency and a failure point before the actual generation API call.
  • The failure is especially painful for short-lived job runners where per-request in-memory metadata cache is cold.

Possible fix direction

Add an explicit-model path that does not require modelMetadataDirectory()->getModelMetadata( $modelId ) to hit a live list-models endpoint.

Possible shapes:

  • Let providers create model metadata from an explicit model ID without listing all models.
  • Add a createModelFromId() / getFallbackModelMetadata()-style provider hook.
  • For OpenAI-compatible providers, derive conservative capabilities/options from known model ID prefixes and let the generation endpoint be authoritative.

The key contract change: getProviderModel( $provider, $modelId ) should not require network I/O merely to instantiate the explicitly requested model.

Related mitigation

Persistent cache via AiClient::setCache() can reduce repeated /models calls, but it does not fix this contract. It still makes explicit model usage depend on successful model-list metadata discovery at least once per cache TTL.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions