OpenAI-Compatible API Compatibility Checker

A local, provider-neutral smoke-test CLI for an OpenAI-compatible endpoint.

It checks model discovery, Chat Completions, Responses, streaming, tool-call requests, structured errors, and whether a Chat response contains usage. It is intentionally a smoke test, not a compatibility certification: agent adapters can require additional configuration and state handling.

Safety model

• Runs on the local machine.
• Does not store results, prompts, or API keys.
• Does not print API keys.
• Sends an optional API key only to the --base-url selected by the operator.
• Uses synthetic short prompts. Do not use it with a production endpoint unless its costs and data handling are acceptable.

Run

export OPENAI_API_KEY='your-key'
node api-compat-check.mjs \
  --base-url https://provider.example/v1 \
  --model your-model-id

Use a different environment variable when needed:

export PROVIDER_TEST_KEY='your-key'
node api-compat-check.mjs \
  --base-url https://provider.example/v1 \
  --model your-model-id \
  --api-key-env PROVIDER_TEST_KEY

Use --json for automation and --timeout-ms 30000 only when the provider's documented latency warrants it. The default output is human-readable. Exit 0 means no request-level check failed; exit 2 means at least one check failed; exit 1 means invalid arguments or a transport error. The tool has no telemetry and never writes credentials, prompts, or results to disk.

Interpret results

• TESTED: the request completed and met this tool's narrow assertion.
• PARTIAL: the endpoint responded, but the required shape was absent.
• FAILED: the endpoint did not meet the request-level check.

A complete coding-agent validation should separately test the real client, provider-specific configuration, tool execution, cancellation, retries, billing, and model behavior.

Test

npm test