SARIF-aligned scan report envelope spec

[aliksir]

Context

Cross-referencing anthropics/skills#492 — discussion on enabling registry-layer trust labels using scanner output as evidence. @ppcvote (prompt-defense-audit / ultraprobe) and skill-security-check agreed to align on a shared envelope so registries can consume any compliant scanner without per-tool branching.

Boundary: scanner-layer spec (envelope, taxonomy) lives here. Registry/attestation policy is downstream and out of scope.

Proposed Minimal Envelope

1. Severity Taxonomy

Tier	Definition
critical	Actively exploitable in default configuration (e.g., arbitrary code execution, data exfiltration)
high	Exploitable with minimal user interaction or configuration (e.g., permission bypass, prompt injection vector)
medium	Requires specific conditions to exploit (e.g., context window poisoning, indirect data leak)
low	Informational or defense-in-depth concern (e.g., missing best practice, verbose error output)

2. SARIF Envelope (minimal fields)

{
  "$schema": "https://raw.githubusercontent.com/oasis-tcs/sarif-spec/main/sarif-2.1/schema/sarif-schema-2.1.0.json",
  "version": "2.1.0",
  "runs": [{
    "tool": {
      "driver": {
        "name": "claude-code-skill-security-check",  // scanner name
        "version": "3.1.1",
        "informationUri": "https://github.com/aliksir/claude-code-skill-security-check",
        "rules": [/* rule definitions with severity tier */]
      }
    },
    "results": [{
      "ruleId": "prompt-injection-vector",
      "level": "error",                              // SARIF level mapped from tier
      "message": { "text": "..." },
      "properties": {
        "severity_tier": "critical",                  // canonical tier from taxonomy above
        "taxonomy_uri": "https://github.com/aliksir/claude-code-skill-security-check/blob/main/TAXONOMY.md"
      }
    }],
    "properties": {
      "target": {
        "name": "skill-name",
        "digest": {
          "sha256": "abc123..."                       // content hash of scanned artifact
        }
      }
    }
  }]
}

3. Severity → SARIF Level Mapping

Tier	SARIF level
critical	error
high	error
medium	warning
low	note

4. Digest Requirements

• Algorithm: sha256 (mandatory), additional algorithms optional
• Scope: hash of the scanned artifact content (e.g., SKILL.md + scripts/ + references/)
• Purpose: bind scan results to specific content version; invalidate on content change

Open Questions

• Should taxonomy_uri point to a shared repo or each scanner's own taxonomy doc?
• Minimum required fields in rules[] — just id + shortDescription + defaultConfiguration.level?
• Should we include a scanStartedUtc / scanCompletedUtc in run properties?

Next Steps

1. Feedback on this draft from @ppcvote and anyone else interested
2. Ship TAXONOMY.md in this repo
3. Implement SARIF output in skill-security-check
4. ppcvote aligns prompt-defense-audit / ultraprobe output

cc @ppcvote

[ppcvote]

@aliksir thanks for the fast turnaround — the envelope shape is clean. Answers to your three open questions:

Q1 (taxonomy_uri target): per-scanner, not shared. Aligns with the framing in agentskills/agentskills#418 — the moment a shared enum becomes spec-blessed, whoever owns it gains gatekeeper status by accident. Per-scanner URI keeps the envelope neutral; consumers decide which taxonomy URIs they trust at policy time. Concrete: skill-security-check points at your TAXONOMY.md, prompt-defense-audit at its own taxonomy doc, downstream consumers list permitted URIs.

Worth allowing taxonomy_uri on a per-result basis too, not just per-rule — lets a single scan emit findings against multiple taxonomies in one run (e.g., one scanner runs both OWASP LLM rules and MITRE ATLAS rules, each result tagged accordingly).

Q2 (minimum rules[] fields): your three are right, plus one optional. id, shortDescription.text, defaultConfiguration.level as required, and helpUri as recommended-but-optional. helpUri is where the rule's full documentation lives and composes cleanly with the taxonomy_uri story — taxonomy_uri identifies the framework, helpUri the specific rule within it. Consumers that want a click-through from a finding to the rule docs get it for free.

Q3 (scan timestamps): yes, but as SARIF native invocations[].startTimeUtc / endTimeUtc, not custom properties.scanStartedUtc. That way the timing flows into GitHub Code Scanning timeline and other SARIF-aware tooling without custom adapters. Standard SARIF 2.1.0 already pins the format and parsing behavior.

One small addition on target.digest: consider including the algorithm name explicitly — digest: { "algorithm": "sha256", "value": "abc123..." } instead of digest: { "sha256": "abc123..." }. Matches OCI image-spec convention and leaves room for sha3-256 / blake3 later without breaking parsers that key on the property name.

Will start mapping prompt-defense-audit + ultraprobe output to this envelope once TAXONOMY.md is up — happy to send a draft mapping for review before flipping the scanner output itself, so we catch any per-tool weirdness before consumers see it.

[aliksir]

Thanks for the thorough answers — all four points land well. Here's where we're at:

Q1: per-scanner taxonomy_uri ✅

Agreed. Per-scanner URI keeps the envelope neutral — no accidental gatekeeper. Each scanner owns its own taxonomy doc, consumers whitelist the URIs they trust at policy time.

On per-result taxonomy_uri: the idea makes sense in principle (one scan emitting findings against OWASP LLM + MITRE ATLAS in a single run). Before baking it in, could you give one concrete use case from prompt-defense-audit or ultraprobe where a single scan run actually tags results against two different taxonomies today? If yes, we add it as a first-class field. If it's forward-looking, we spec it as MAY (optional) so the envelope stays lean for v1.

Separately — SARIF's native approach for multi-taxonomy results is result.taxa[] + run.taxonomies[] rather than custom properties. Worth considering whether we use the standard path instead of a custom taxonomy_uri on each result, so GitHub Code Scanning and other SARIF-aware consumers get it for free.

Q2: minimum rules[] fields ✅

Locked in:

• Required: id, shortDescription.text, defaultConfiguration.level
• Recommended: helpUri (standard SARIF field on rules[], not in properties)

Q3: scan timestamps ✅

Agreed — invocations[].startTimeUtc / endTimeUtc is the right call over custom properties.

One addition: SARIF 2.1.0 requires executionSuccessful as a mandatory field on invocations[]. We need to include it:

"invocations": [{
  "startTimeUtc": "2026-06-25T00:00:00Z",
  "endTimeUtc": "2026-06-25T00:01:23Z",
  "executionSuccessful": true
}]

Digest format ✅

Good call on explicit algorithm name. Adopting OCI image-spec convention:

"digest": {
  "algorithm": "sha256",
  "value": "abc123..."
}

Leaves room for blake3/sha3-256 without breaking parsers.

Next steps

1. I'll ship TAXONOMY.md in this repo with the severity tiers + updated envelope spec incorporating all the above
2. Send your draft mapping for prompt-defense-audit + ultraprobe whenever ready — review before flipping scanner output sounds right

[ppcvote]

Three quick responses, then I think we're aligned enough to ship the envelope draft.

Per-result taxonomy_uri — honest answer: MAY for v1

Checked both my scanners. Neither emits multi-taxonomy results today:

• prompt-defense-audit: internal vector IDs only, no OWASP-LLM / MITRE ATLAS / ATR tagging in output, no SARIF emitter shipped yet.
• ultraprobe (npm v2.1.1): emits valid SARIF 2.1.0 from the CLI, but only populates tool.driver.rules[] with its own namespace. No result.taxa[], no run.taxonomies[]. One ruleId per result. (Current-state honesty, not a deprecation of the idea — happy to wire it once the envelope lands.)

So per-result taxonomy binding is forward-looking, not exercised. First real user will be a multi-taxonomy mapping run (e.g. OWASP LLM + MITRE ATLAS + ATR from a single scan). Not exercised on my side today. Spec it as MAY/optional for v1, lock in once a concrete consumer hits it.

SARIF-native result.taxa[] + run.taxonomies[] — yes

Agreed, use the standard path. result.taxa[] referencing run.taxonomies[].taxa[] is what SARIF 2.1.0 defines for exactly this case. Custom taxonomy_uri properties would lose us the existing tooling that already understands the native shape. Envelope should normalize on the SARIF approach and document the mapping rather than reinvent.

executionSuccessful — good catch

You're right, invocations[].executionSuccessful is required by SARIF 2.1.0. Adding it to the envelope spec alongside startTimeUtc / endTimeUtc. Failed scans still emit a valid envelope with executionSuccessful: false + toolExecutionNotifications[] for the failure reason.

Ship the envelope issue when ready — happy to review against my SARIF emitter as a reference implementation and flag any gaps from the scanner side.

[aliksir]

Thanks — all three points are aligned, nothing to push back on.

The envelope RFC is up at #24 covering the agreed specs:

• per-result taxonomy_uri (MAY / optional)
• SARIF-native result.taxa[] + run.taxonomies[]
• invocations[].executionSuccessful

Skeleton modules (draft-validator, alias-map, draft-tracker) are implemented with 39 passing tests.

Would appreciate your review against your SARIF emitter when you get a chance.