Summary
A skill containing a single 85-byte PNG scores 55/100, HIGH, DO_NOT_INSTALL with --no-llm, despite having no scripts, no executables, and eight lines of markdown. Three findings fire, two of them against the PNG's decoded bytes.
The notable part is that the same report already knows the file is unreadable. The completeness ledger records:
Ledger exceptions
- opaque_content assets/diagram.png: Artifact contents could not be fully interpreted.
So one subsystem classifies the artifact as opaque and excludes it, while AE3 and AE4 analyze its decoded content anyway and report findings at 90% and 80% confidence. Those two conclusions cannot both be right.
This is the same class as #144 (closed COMPLETED 2026-06-24), which covered binary content reaching static pattern analyzers via MP2. The MP2 path appears fixed; the AE* analyzers were not covered by that fix.
Environment
- SkillSpector 2.9.6, installed from
main at 698e2bf29c7d32aa8211ada677382460c01900d7
- Windows 11, Python 3.14,
--no-llm (static only)
Minimal reproduction
mkdir -p binary-repro/assets
cat > binary-repro/SKILL.md <<'EOF'
---
name: binary-repro
description: A skill that ships one small PNG as reference material.
---
# Binary repro
Describe the diagram in assets/diagram.png to the user.
EOF
# 85-byte 1x1 PNG
python3 -c "import base64; open('binary-repro/assets/diagram.png','wb').write(base64.b64decode('iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKCAYAAACNMs+9AAAAHElEQVQoz2NgGAWjYBSMglEwCkbBKBgFo2AUAAAHkgABfXRPtQAAAABJRU5ErkJggg=='))"
skillspector scan binary-repro --no-llmActual result
Risk Assessment
Score 55/100
Severity HIGH
Recommendation DO NOT INSTALL
Issues (3)
HIGH: AE1 - Referenced artifact was not completely inspected... SKILL.md:8 conf 100%
HIGH: AE3 - Text artifact contains embedded NUL bytes... assets/diagram.png:3 conf 90%
MEDIUM: AE4 - Suspicious Unicode normalization or mixed-script... assets/diagram.png:1 conf 80%
Inspection Completeness
Coverage 50.0%
Entirely uninspected 1
Ledger exceptions
- opaque_content assets/diagram.png: Artifact contents could not be fully interpreted.
Fifteen analyzers additionally report degraded under Limitations.
Note the component inventory lists assets/diagram.png as Lines: 8 — the PNG is being line-counted as text, which is where the NUL-byte and mixed-script observations come from.
Expected result
A PNG shipped as reference material is not a security finding. Specifically:
- An artifact the ledger classifies as
opaque_content should be excluded from content analyzers, not analyzed and reported on. Whatever check produces the ledger exception should gate the analyzers, so the two cannot disagree.
AE3 "embedded NUL bytes" is a tautology for any binary format. NUL bytes are evidence a file is binary, not evidence it is malicious.AE4 mixed-script detection over bytes that were never text has no meaningful signal.- If
AE1 is intended to flag genuinely uninspectable references, a recognized binary asset type is not the case it should fire HIGH at 100% confidence on.
Impact
Severity drives the score, so a single HIGH sets the verdict regardless of finding count. Any skill shipping a screenshot, diagram, or icon — a common and entirely legitimate pattern — inherits DO_NOT_INSTALL. For anyone using the MCP server to gate installs (scan_skill → safe_to_install), this blocks benign skills automatically.
It also degrades the tool's signal in the direction that matters most: users who see DO_NOT_INSTALL on a skill they know is fine learn to discount the verdict, which is the failure mode #144's original report described.
General rule
Worth applying beyond these two pattern IDs: an analyzer should never emit a content finding about an artifact the inspection ledger has already classified as uninterpretable. The ledger is the tool's own record of what it could and could not read — treating it as authoritative for analyzer applicability would close this whole class in one place, rather than per-pattern as it is discovered.
That framing may also cover #367 (incomplete skip denylist, CWE-184) and relates to #389 (coverage computed but never affecting the exit code): coverage, ledger exceptions, and analyzer applicability are three views of the same question, and they currently disagree.
Related
Happy to test a fix against this repro, or against a wider corpus of skills shipping images if that is useful.
Summary
A skill containing a single 85-byte PNG scores 55/100, HIGH,
DO_NOT_INSTALLwith--no-llm, despite having no scripts, no executables, and eight lines of markdown. Three findings fire, two of them against the PNG's decoded bytes.The notable part is that the same report already knows the file is unreadable. The completeness ledger records:
So one subsystem classifies the artifact as opaque and excludes it, while
AE3andAE4analyze its decoded content anyway and report findings at 90% and 80% confidence. Those two conclusions cannot both be right.This is the same class as #144 (closed COMPLETED 2026-06-24), which covered binary content reaching static pattern analyzers via
MP2. TheMP2path appears fixed; theAE*analyzers were not covered by that fix.Environment
mainat698e2bf29c7d32aa8211ada677382460c01900d7--no-llm(static only)Minimal reproduction
Actual result
Fifteen analyzers additionally report
degradedunder Limitations.Note the component inventory lists
assets/diagram.pngasLines: 8— the PNG is being line-counted as text, which is where the NUL-byte and mixed-script observations come from.Expected result
A PNG shipped as reference material is not a security finding. Specifically:
opaque_contentshould be excluded from content analyzers, not analyzed and reported on. Whatever check produces the ledger exception should gate the analyzers, so the two cannot disagree.AE3"embedded NUL bytes" is a tautology for any binary format. NUL bytes are evidence a file is binary, not evidence it is malicious.AE4mixed-script detection over bytes that were never text has no meaningful signal.AE1is intended to flag genuinely uninspectable references, a recognized binary asset type is not the case it should fire HIGH at 100% confidence on.Impact
Severity drives the score, so a single HIGH sets the verdict regardless of finding count. Any skill shipping a screenshot, diagram, or icon — a common and entirely legitimate pattern — inherits
DO_NOT_INSTALL. For anyone using the MCP server to gate installs (scan_skill→safe_to_install), this blocks benign skills automatically.It also degrades the tool's signal in the direction that matters most: users who see
DO_NOT_INSTALLon a skill they know is fine learn to discount the verdict, which is the failure mode #144's original report described.General rule
Worth applying beyond these two pattern IDs: an analyzer should never emit a content finding about an artifact the inspection ledger has already classified as uninterpretable. The ledger is the tool's own record of what it could and could not read — treating it as authoritative for analyzer applicability would close this whole class in one place, rather than per-pattern as it is discovered.
That framing may also cover #367 (incomplete skip denylist, CWE-184) and relates to #389 (coverage computed but never affecting the exit code): coverage, ledger exceptions, and analyzer applicability are three views of the same question, and they currently disagree.
Related
MP2(closed COMPLETED, same class, different analyzers).git/hooks/*.sample; this repro deliberately has no git repositoryHappy to test a fix against this repro, or against a wider corpus of skills shipping images if that is useful.