fix(cli): dry-run guards vision/text-only attachment conflict (#536)#542
Merged
Conversation
`panel run --dry-run` reported "Validation: OK" even when the resolved instrument carried image/document/screenshot attachments while the selected model is text-only. A real run fast-fails on the same condition via assert_supports_attachments, so dry-run gave false confidence and let the user proceed to burn tokens on a no-image response. The dry-run preview now scans the instrument's attachments (shared bank + per-question inline attachments) for visual kinds — `image` (ImageBlock), `document` (DocumentBlock), and `url` with `fetch_mode: screenshot` (lowers to an image) — and, when the model is text-only (model_supports_vision), emits "Validation: WARNING" in text mode and `validation: "warning"` plus a structured `vision_conflict` block in JSON mode, instead of "OK". Adds tests: image attachment + text-only model warns (text + JSON), screenshot url + text-only model warns, and a vision-capable model still validates OK. Closes #536 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
claude-dataviking
requested review from
openclaw-dv and
the-data-viking
as code owners
Deploying synthpanel with
|
| Latest commit: |
134deab
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://20df3c69.synthpanel.pages.dev |
| Branch Preview URL: | https://fix-dry-run-vision-guard.synthpanel.pages.dev |
the-data-viking
approved these changes
Jun 3, 2026
the-data-viking
added a commit
that referenced
this pull request
Jun 3, 2026
Cuts the patch release containing the four bug fixes merged today (#540 cost hint, #541 mcp-install path, #542 dry-run vision guard, #543 report synthesize hint) plus the attachments docs (#535). Pre-bumps __version__.py and re-renders the version artifacts (render_site.py, render_site_markdown.py, render_server_card.py) so auto-tag.yml hits its "nothing to commit" path and only tags v1.5.6 — its direct version-bump push to main is currently rejected by branch protection (GH013), which is why applying semver:patch to a regular fix PR (#543) failed to release. Filing that pipeline issue separately. Co-authored-by: Wesley Johnson <wesley@dataviking.tech> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
panel run --dry-runreportedValidation: OKeven when the resolved instrument carried image/document/screenshot attachments while the selected model is text-only. A real run fast-fails on exactly that condition viaassert_supports_attachments(src/synth_panel/llm/capabilities.py), so the dry-run gave false confidence and let the user proceed to a token-burning no-image run.The dry-run preview now scans the instrument's attachments — the shared bank (
Instrument.attachments) and per-question inline attachments — for visual kinds:image-> ImageBlockdocument-> DocumentBlockurlwithfetch_mode: screenshot(lowers to an image block)When a visual attachment is present and
model_supports_vision(model)is False, the preview emits:Validation: WARNING — <reason>instead ofValidation: OKvalidation: "warning"plus a structuredvision_conflictblock (model,attachment_kinds,message)Exit code stays 0 (it's a preview/warning, matching the existing dry-run contract); the message points at vision-capable alternatives.
Tests (in
tests/test_cli.py)test_dry_run_text_only_model_with_image_attachment_warnstest_dry_run_text_only_model_with_image_attachment_jsontest_dry_run_vision_model_with_image_attachment_ok(no false positive)test_dry_run_text_only_model_with_screenshot_url_warnsGates
ruff check ./ruff format --check .: passpytest tests/test_cli.py tests/test_vision_capability.py tests/attachments: 232 passedCloses #536
🤖 Generated with Claude Code