Agent Skill Doctor

Static quality and security checks for SKILL.md-based Agent Skills. Zero runtime dependencies, CI-ready, and safe by default.

简体中文 · Contributing · Security

Agent Skill Doctor catches incomplete metadata, broken references, packaging mistakes, and risky instructions before a Skill reaches users. It only reads target files: it never imports or executes scripts from the Skill being audited.

v0.1 is an alpha release. Rules and report fields may evolve before a versioned schema is introduced.

30-second demo

$ skill-doctor path/to/untrusted-skill --fail-on error
Agent Skill Doctor 0.1.0
Skills: 1 | critical: 1 | error: 2 | warning: 1 | info: 1

untrusted-skill (.)
  [CRITICAL] SEC001 SKILL.md:24
    A downloaded payload is piped directly into a shell.
    Fix: Download to a reviewed file, verify integrity, and execute only after explicit approval.

  [ERROR] LINK001 SKILL.md:12
    Reference 'references/setup.md' does not exist.

  [ERROR] SEC002 scripts/install.sh:8
    Recursive forced deletion is present.

$ echo $?
1

The output is designed to answer four questions immediately: what failed, how serious it is, where the evidence is, and what to change.

Highlights

• Scan a SKILL.md file, a single Skill directory, or every Skill under a repository.
• Validate frontmatter, naming, descriptions, local links, package hygiene, and Agent metadata.
• Detect common remote-execution, destructive-command, sensitive-path, dynamic-execution, network, encoding, and symlink risks.
• Produce terminal text, machine-readable JSON, or GitHub-compatible SARIF.
• Select the CI failure threshold with --fail-on.
• Run as a Python CLI, a self-contained bundled script, or a composite GitHub Action.
• Refuse to read files through symlinks that escape the audited Skill directory.
• Redact common credentials from evidence snippets.

Installation

Agent Skill Doctor requires Python 3.10 or newer.

Install the tagged release from GitHub

python -m pip install "git+https://github.com/014-code/agent-skill-doctor.git@v0.1.0"
skill-doctor path/to/my-skill

Clone for development or Skill installation

git clone https://github.com/014-code/agent-skill-doctor.git
cd agent-skill-doctor
python -m pip install --editable .
skill-doctor skills/skill-doctor --fail-on warning

The reusable Codex Skill is bundled at skills/skill-doctor. The scanner can also run without installation:

python skills/skill-doctor/scripts/skill_doctor.py path/to/my-skill

CLI

skill-doctor PATH [--format text|json|sarif] [--output FILE]
                  [--fail-on error|warning|critical|never]
                  [--no-security] [--version]

Option	Meaning
PATH	A SKILL.md file, Skill directory, or repository tree containing Skills.
--format	text for terminals, json for automation, or sarif for code scanning.
--output FILE	Write the report to a file instead of standard output.
--fail-on	Fail at this severity or above. The default is error; warning is the strictest gate.
--no-security	Skip heuristic content and script security rules. Structural boundary checks still apply.
--version	Print the installed version.

Exit code 0 means no finding reached the selected threshold. Exit code 1 means at least one finding reached it. Report generation completes before the exit code is returned.

What v0.1 checks

Rule family	Default severity	Checks
PATH, SKILL, FM	error / warning	Skill discovery, readable SKILL.md, and valid frontmatter.
NAME, DESC, BODY	error / warning	Naming, trigger descriptions, template remnants, and context size.
LINK	error / warning	Broken, absolute, or directory-escaping local references.
AGENT	warning	agents/openai.yaml interface metadata used by OpenAI/Codex packaging.
PACKAGE, REF, SCAN	warning / info	Package hygiene, long references, and unscannable text resources.
SEC001	critical	Remote content executed directly by a shell or interpreter.
SEC002	error	Destructive deletion, Git cleanup, formatting, or raw-device writes.
SEC003–SEC006	warning / info	Sensitive paths, dynamic execution, network access, or encoded execution.
SEC007	error	Symlinks that escape the Skill directory.

See the complete rule catalog for interpretation and remediation guidance.

Compatibility scope

The core SKILL.md, Markdown reference, package, and security checks are platform-neutral. The AGENT001–AGENT004 rules specifically validate OpenAI/Codex agents/openai.yaml metadata. Other Agent platforms can treat those warnings as advisory; the default --fail-on error gate does not fail on them.

GitHub Action

name: Check Agent Skills

on:
  pull_request:
  push:

jobs:
  skill-doctor:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: 014-code/agent-skill-doctor@v0.1.0
        with:
          path: skills
          format: text
          fail-on: error

Generate and upload SARIF to GitHub code scanning:

name: Agent Skill Security

on:
  pull_request:
  push:

permissions:
  contents: read
  security-events: write

jobs:
  audit:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: 014-code/agent-skill-doctor@v0.1.0
        with:
          path: skills
          format: sarif
          output: skill-doctor.sarif
          fail-on: never
      - uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: skill-doctor.sarif

GitHub may restrict security-events: write for pull requests from forks. Keep the normal text audit in CI even when SARIF upload is unavailable.

Safety model and limitations

• The scanner performs static, heuristic analysis. Findings are review leads, not proof of malicious intent.
• It is not a sandbox, malware scanner, prompt-injection detector, or runtime behavior monitor.
• False positives and false negatives are possible, especially for dynamically constructed commands.
• Target scripts are never imported or executed.
• External symlink targets are reported but not read.
• Text files larger than 1 MB are skipped and reported as SCAN001.
• v0.1 understands the common SKILL.md layout and OpenAI/Codex metadata; platform-specific profiles are planned.

Use a sandboxed forward test in addition to static analysis before trusting a complex third-party Skill.

Roadmap

• v0.2: auditable inline suppressions, configurable rule profiles, confidence scores, and improved source-to-sink checks.
• v0.x: Claude Code and other Agent metadata profiles, baselines for existing findings, and richer SARIF properties.
• v1: versioned JSON schema, documented rule SDK, compatibility guarantees, and stable CI policy behavior.

Development

python -m pip install --editable ".[dev]"
python -m pytest
python -m build
skill-doctor skills/skill-doctor --fail-on warning

The scanner uses only the Python standard library at runtime. Tests create temporary Skill fixtures and never execute fixture instructions.

Contributions are welcome. Read CONTRIBUTING.md before opening a pull request, and report vulnerabilities through the process in SECURITY.md.

License

MIT