Skip to content

bug: Anthropic silently converts URL images to placeholder text — lossy and undocumented #64

@stackbilt-admin

Description

@stackbilt-admin

Bug

When a message contains an image specified by URL (rather than base64), the Anthropic provider silently replaces it with a text placeholder instead of throwing an error.

Location: src/providers/anthropic.ts:498–501

if (image.url) {
  return {
    type: 'text' as const,
    text: `[Image URL: ${image.url}]`
  };
}

The Anthropic native API does not support image content blocks with a URL source — only base64. This conversion is technically correct in that it avoids an API error, but it is lossy: the image is not processed at all. The caller receives a successful response from what is effectively a text-only request.

Why this is wrong

  • The LLMImageInput interface (src/types.ts) accepts both data (base64) and url — consumers reasonably expect URL images to work or fail explicitly
  • There is no warning or error surfaced to the caller
  • The behavior differs silently from OpenAI (which accepts image URLs natively)

Fix

Replace the silent conversion with a ConfigurationError:

if (image.url) {
  throw new ConfigurationError(
    'Anthropic does not support image URLs — convert to base64 before sending, ' +
    'or use an OpenAI-compatible provider for URL-based vision.'
  );
}

Alternatively, auto-fetch the URL and convert to base64 in the provider — but explicit failure is preferable so callers know what's happening.

Acceptance criteria

  • Passing a URL image to AnthropicProvider throws ConfigurationError (or auto-fetches and converts — decision left to implementor)
  • Behavior is documented in a JSDoc comment on the image-handling path
  • Test added for the URL image path on Anthropic

Found by

Codebase audit (automated) — src/providers/anthropic.ts:498–501

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions