Bundled agents: 'auto' model + surface silent model substitution#84
Merged
Conversation
The bundled demo agents shipped `model: 'claude-haiku-4-5'`, but no provider exposes that exact id — wp-ai-client silently falls back to the provider's default (Opus on Anthropic), so the demos ran on an expensive model while the config implied Haiku, with no signal. - Switch the bundled agents (example, loop-demo, site-introspection, workflow-drafter, coordinator) to `model: 'auto'` so they adopt whatever the active provider is configured for, instead of a misleading, unresolvable id. - Runner: when an agent pins a model id that the provider substitutes, record it on the turn telemetry and log `[openclawp] model_substitution agent=… requested=… used=…`, so silent fallback to a premium model is visible. - README: document that cloud defaults pick the provider's (often priciest) model, how to pin a specific id, and the substitution log line. Note: 'auto' still defers model choice to the provider (Opus by default on Anthropic). Truly preventing the silent premium fallback is the provider / wp-ai-client's job — filed separately upstream. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The bundled agents now declare model 'auto' instead of the unresolvable 'claude-haiku-4-5'; regenerate the five payload snapshots to match. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
resolve_model_preference() returns null for model 'auto', so the payload's model_preference is null (not an object). Correct the five snapshots. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The bundled demo agents shipped
model: 'claude-haiku-4-5'. No provider exposes that exact id (Anthropic's isclaude-haiku-4-5-20251001), so wp-ai-client silently falls back to the provider's default —claude-opus-4-8on Anthropic. The demos ran on the priciest model while the config implied Haiku, with zero signal. Confirmed live: aclaude-haiku-4-5agent reportsmodel:claude-opus-4-8in telemetry.Changes
model: 'auto'(example, loop-demo, site-introspection, workflow-drafter, coordinator). They now adopt whatever the active provider is configured for, rather than a misleading, unresolvable id.model_requestedand the runner logs:Honest scope
'auto'still defers to the provider's default (Opus on Anthropic) — this PR removes the misleading id and makes pinned-model substitution visible, but it doesn't force a cheap cloud default. Genuinely preventing the silent premium fallback is the provider / wp-ai-client's responsibility (filed upstream separately). Pair with the per-site budget caps.🤖 Generated with Claude Code