Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,7 @@ clawscan scanners skillspector
| `agentverus` | AgentVerus | [repo](https://github.com/agentverus/agentverus-scanner) | Local file or directory scanner invoked through agentverus-scanner. | none | `npm install --save-dev agentverus-scanner` |
| `aig` | Tencent AI-Infra-Guard | [repo](https://github.com/Tencent/AI-Infra-Guard/tree/main/skill-scan) | Tencent Zhuque Lab's local directory scanner invoked through `aig-skill-scan`. Produces SARIF 2.1.0 with SkillTrustBench T01-T09 evidence. | `LLM_API_KEY` or `OPENAI_API_KEY`<br><details><summary>Optional config</summary><code>DEFAULT_MODEL</code>, <code>DEFAULT_BASE_URL</code>, <code>DEFAULT_MODEL_CONTEXT_WINDOW</code>, <code>LOG_LEVEL</code>.</details> | `pip install aig-skill-scan` |
| `cisco` | Cisco AI Defense skill-scanner | [repo](https://github.com/cisco-ai-defense/skill-scanner) | Local file or directory scanner invoked through `skill-scanner` with JSON report output. Optional upstream env vars enable LLM, VirusTotal, and Cisco AI Defense analyzers. | none<br><details><summary>Optional config</summary><code>SKILL_SCANNER_LLM_API_KEY</code>, <code>SKILL_SCANNER_LLM_PROVIDER</code>, <code>SKILL_SCANNER_LLM_MODEL</code>, <code>SKILL_SCANNER_LLM_BASE_URL</code>, <code>SKILL_SCANNER_LLM_USER</code>, <code>SKILL_SCANNER_LLM_API_VERSION</code>, <code>SKILL_SCANNER_LLM_FORCE_JSON_OBJECT</code>, <code>SKILL_SCANNER_META_LLM_API_KEY</code>, <code>SKILL_SCANNER_META_LLM_MODEL</code>, <code>SKILL_SCANNER_META_LLM_BASE_URL</code>, <code>SKILL_SCANNER_META_LLM_API_VERSION</code>, <code>AWS_PROFILE</code>, <code>AWS_REGION</code>, <code>GOOGLE_APPLICATION_CREDENTIALS</code>, <code>VIRUSTOTAL_API_KEY</code>, <code>AI_DEFENSE_API_KEY</code>, <code>AI_DEFENSE_API_URL</code>.</details> | `uv pip install cisco-ai-skill-scanner` |
| `clawscan-static` | ClawScan Static | [repo](https://github.com/openclaw/clawscan) | Built-in deterministic text scanner for high-signal risky skill and OpenClaw plugin patterns. | none | skipped; built in |
| `clawscan-static` | ClawScan Static | [repo](https://github.com/openclaw/clawscan) | Built-in deterministic scanner for high-signal risky skill and OpenClaw plugin patterns; packaged Python bytecode is flagged as opaque executable content. | none | skipped; built in |
| `relyable` | Relyable | [repo](https://github.com/veriker/relyable) | Functional re-derivation evidence: does the skill still do what its docs claim, recomputed? Emits the strongest grade that applies. `exogenous`: a declared `rederive.json` property manifest (idempotence / round-trip), with both sides of the relation computed from the skill's own code and the result mutation-tested against vacuity. `self_spec`: re-runs the author's own committed oracle (shipped tests or documented I/O examples). `cold_golden`: when an LLM key is set, a code-blind model infers goldens from SKILL.md alone and abstains unless the docs pin exact behavior; divergences are reported as unconfirmed, never as accusations. `non_rederivable`: the honest floor, never a fabricated pass. Functional axis only; complements the security scanners and does not detect malware or prompt injection. Skill code runs only inside the Docker sandbox (or with an explicit opt-in), in a scrubbed environment, and the scanner fails closed otherwise. Not preinstalled in the `clawscan-runtime` image. | none<br><details><summary>Optional config</summary><code>RELYABLE_SCAN_ALLOW_HOST_EXEC</code> — explicit ack that the host is disposable when running with <code>--sandbox off</code>.<br><br><code>RELYABLE_LLM_API_KEY</code> (+ <code>RELYABLE_LLM_PROVIDER</code> <code>anthropic|openai</code>, <code>RELYABLE_LLM_MODEL</code>, <code>RELYABLE_LLM_BASE_URL</code>) — explicit per-scanner opt-in that enables the <code>cold_golden</code> lane; key presence only is ever recorded in the payload. Generic <code>ANTHROPIC_API_KEY</code>/<code>OPENAI_API_KEY</code> are honored by standalone <code>relyable-scan</code> but are deliberately not auto-forwarded by ClawScan.</details> | `clawscan install relyable` — not preinstalled in the runtime image |
| `skillspector` | NVIDIA SkillSpector | [repo](https://github.com/NVIDIA/skillspector) | Local skill or OpenClaw plugin file/directory scanner. Uses LLM mode when provider env vars are set; otherwise runs with `--no-llm`. | none<br><details><summary>Optional config</summary><code>SKILLSPECTOR_PROVIDER</code>, <code>SKILLSPECTOR_MODEL</code>, <code>SKILLSPECTOR_MODEL_REGISTRY</code>, <code>SKILLSPECTOR_LOG_LEVEL</code>, <code>SKILLSPECTOR_SSL_VERIFY</code>, <code>NVIDIA_INFERENCE_KEY</code>, <code>OPENAI_API_KEY</code>, <code>OPENAI_BASE_URL</code>, <code>ANTHROPIC_API_KEY</code>, <code>ANTHROPIC_PROXY_ENDPOINT_URL</code>, <code>ANTHROPIC_PROXY_API_KEY</code>, <code>ANTHROPIC_PROXY_API_VERSION</code>.</details> | `uv tool install git+https://github.com/NVIDIA/skillspector.git` |
| `snyk` | Snyk Agent Scan | [repo](https://github.com/snyk/agent-scan) | Local skill scanner invoked through `uvx snyk-agent-scan`. | `SNYK_TOKEN` | verifies `uvx` launcher |
Expand Down
2 changes: 1 addition & 1 deletion docs/scanners.md
Original file line number Diff line number Diff line change
Expand Up @@ -230,7 +230,7 @@ commas, single-quoted strings, and unquoted keys.
| `agentverus` | AgentVerus | [repo](https://github.com/agentverus/agentverus-scanner) | Local file or directory scanner invoked through agentverus-scanner. | none | `npm install --save-dev agentverus-scanner` |
| `aig` | Tencent AI-Infra-Guard | [repo](https://github.com/Tencent/AI-Infra-Guard/tree/main/skill-scan) | Tencent Zhuque Lab's local directory scanner invoked through `aig-skill-scan`. Produces SARIF 2.1.0 with SkillTrustBench T01-T09 evidence. | `LLM_API_KEY` or `OPENAI_API_KEY`<br><details><summary>Optional config</summary><code>DEFAULT_MODEL</code>, <code>DEFAULT_BASE_URL</code>, <code>DEFAULT_MODEL_CONTEXT_WINDOW</code>, <code>LOG_LEVEL</code>.</details> | `pip install aig-skill-scan` |
| `cisco` | Cisco AI Defense skill-scanner | [repo](https://github.com/cisco-ai-defense/skill-scanner) | Local file or directory scanner invoked through `skill-scanner` with JSON report output. Optional upstream env vars enable LLM, VirusTotal, and Cisco AI Defense analyzers. | none<br><details><summary>Optional config</summary><code>SKILL_SCANNER_LLM_API_KEY</code>, <code>SKILL_SCANNER_LLM_PROVIDER</code>, <code>SKILL_SCANNER_LLM_MODEL</code>, <code>SKILL_SCANNER_LLM_BASE_URL</code>, <code>SKILL_SCANNER_LLM_USER</code>, <code>SKILL_SCANNER_LLM_API_VERSION</code>, <code>SKILL_SCANNER_LLM_FORCE_JSON_OBJECT</code>, <code>SKILL_SCANNER_META_LLM_API_KEY</code>, <code>SKILL_SCANNER_META_LLM_MODEL</code>, <code>SKILL_SCANNER_META_LLM_BASE_URL</code>, <code>SKILL_SCANNER_META_LLM_API_VERSION</code>, <code>AWS_PROFILE</code>, <code>AWS_REGION</code>, <code>GOOGLE_APPLICATION_CREDENTIALS</code>, <code>VIRUSTOTAL_API_KEY</code>, <code>AI_DEFENSE_API_KEY</code>, <code>AI_DEFENSE_API_URL</code>.</details> | `uv pip install cisco-ai-skill-scanner` |
| `clawscan-static` | ClawScan Static | [repo](https://github.com/openclaw/clawscan) | Built-in deterministic text scanner for high-signal risky skill and OpenClaw plugin patterns. | none | skipped; built in |
| `clawscan-static` | ClawScan Static | [repo](https://github.com/openclaw/clawscan) | Built-in deterministic scanner for high-signal risky skill and OpenClaw plugin patterns; packaged Python bytecode is flagged as opaque executable content. | none | skipped; built in |
| `relyable` | Relyable | [repo](https://github.com/veriker/relyable) | Functional re-derivation evidence: does the skill still do what its docs claim, recomputed? Emits the strongest grade that applies. `exogenous`: a declared `rederive.json` property manifest (idempotence / round-trip), with both sides of the relation computed from the skill's own code and the result mutation-tested against vacuity. `self_spec`: re-runs the author's own committed oracle (shipped tests or documented I/O examples). `cold_golden`: when an LLM key is set, a code-blind model infers goldens from SKILL.md alone and abstains unless the docs pin exact behavior; divergences are reported as unconfirmed, never as accusations. `non_rederivable`: the honest floor, never a fabricated pass. Functional axis only; complements the security scanners and does not detect malware or prompt injection. Skill code runs only inside the Docker sandbox (or with an explicit opt-in), in a scrubbed environment, and the scanner fails closed otherwise. Not preinstalled in the `clawscan-runtime` image. | none<br><details><summary>Optional config</summary><code>RELYABLE_SCAN_ALLOW_HOST_EXEC</code> — explicit ack that the host is disposable when running with <code>--sandbox off</code>.<br><br><code>RELYABLE_LLM_API_KEY</code> (+ <code>RELYABLE_LLM_PROVIDER</code> <code>anthropic|openai</code>, <code>RELYABLE_LLM_MODEL</code>, <code>RELYABLE_LLM_BASE_URL</code>) — explicit per-scanner opt-in that enables the <code>cold_golden</code> lane; key presence only is ever recorded in the payload. Generic <code>ANTHROPIC_API_KEY</code>/<code>OPENAI_API_KEY</code> are honored by standalone <code>relyable-scan</code> but are deliberately not auto-forwarded by ClawScan.</details> | `clawscan install relyable` — not preinstalled in the runtime image |
| `skillspector` | NVIDIA SkillSpector | [repo](https://github.com/NVIDIA/skillspector) | Local skill or OpenClaw plugin file/directory scanner. Uses LLM mode when provider env vars are set; otherwise runs with `--no-llm`. | none<br><details><summary>Optional config</summary><code>SKILLSPECTOR_PROVIDER</code>, <code>SKILLSPECTOR_MODEL</code>, <code>SKILLSPECTOR_MODEL_REGISTRY</code>, <code>SKILLSPECTOR_LOG_LEVEL</code>, <code>SKILLSPECTOR_SSL_VERIFY</code>, <code>NVIDIA_INFERENCE_KEY</code>, <code>OPENAI_API_KEY</code>, <code>OPENAI_BASE_URL</code>, <code>ANTHROPIC_API_KEY</code>, <code>ANTHROPIC_PROXY_ENDPOINT_URL</code>, <code>ANTHROPIC_PROXY_API_KEY</code>, <code>ANTHROPIC_PROXY_API_VERSION</code>.</details> | `uv tool install git+https://github.com/NVIDIA/skillspector.git` |
| `snyk` | Snyk Agent Scan | [repo](https://github.com/snyk/agent-scan) | Local skill scanner invoked through `uvx snyk-agent-scan`. | `SNYK_TOKEN` | verifies `uvx` launcher |
Expand Down
10 changes: 3 additions & 7 deletions internal/runner/runner.go
Original file line number Diff line number Diff line change
Expand Up @@ -1001,7 +1001,7 @@ func renderTargetFiles(target string) (string, error) {
return nil
}
if entry.IsDir() {
if shouldSkipTargetPath(target, path) {
if shouldSkipTargetDirectory(target, path) {
precomputedOmissions = append(precomputedOmissions, targetFileOmission{path: path, reason: "skipped path"})
return filepath.SkipDir
}
Expand Down Expand Up @@ -1040,10 +1040,6 @@ func renderTargetFiles(target string) (string, error) {
suppressedOmissions++
}
for _, path := range paths {
if shouldSkipTargetPath(target, path) {
addOmission(path, "skipped path")
continue
}
info, err := os.Stat(path)
if err != nil {
return "", err
Expand Down Expand Up @@ -1126,14 +1122,14 @@ func markdownPathLabel(label string) string {
return quoted[1 : len(quoted)-1]
}

func shouldSkipTargetPath(root string, path string) bool {
func shouldSkipTargetDirectory(root string, path string) bool {
rel, err := filepath.Rel(root, path)
if err != nil {
return true
}
for _, part := range strings.Split(rel, string(filepath.Separator)) {
switch part {
case ".git", "node_modules", "vendor", ".venv", "__pycache__":
case ".git", "node_modules", "vendor", ".venv":
return true
}
}
Expand Down
86 changes: 86 additions & 0 deletions internal/runner/runner_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -3078,6 +3078,92 @@ func TestStaticScannerFindsDestructiveRmWithForceBeforeRecursive(t *testing.T) {
t.Fatalf("missing destructive shell finding: %#v", report.Findings)
}

func TestStaticScannerScansGitFileAndFindsProcessExecution(t *testing.T) {
dir := t.TempDir()
target := filepath.Join(dir, "skill")
if err := os.Mkdir(target, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(target, "SKILL.md"), []byte("# Demo\n"), 0o644); err != nil {
t.Fatal(err)
}
payload := "import os\nos.system(\"whoami > pwn\")\n"
if err := os.WriteFile(filepath.Join(target, ".git"), []byte(payload), 0o644); err != nil {
t.Fatal(err)
}
opts, err := ParseArgs([]string{target, "--scanner", "clawscan-static"})
if err != nil {
t.Fatal(err)
}
artifact, err := Run(opts, RunContext{Env: map[string]string{}})
if err != nil {
t.Fatal(err)
}
report := decodeStaticReport(t, artifact.Scanners["clawscan-static"].Raw)
for _, finding := range report.Findings {
if finding.ID == "static.python_process_execution" && finding.Path == ".git" {
prompt, err := RenderPromptTemplate("{{ target.files }}", artifact)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(prompt, "### .git\n```text\n"+strings.TrimSpace(payload)) {
t.Fatalf("judge prompt omitted .git file contents: %s", prompt)
}
return
}
}
t.Fatalf("missing .git process-execution finding: %#v", report.Findings)
}

func TestPythonBytecodeEvidenceReachesJudgePrompt(t *testing.T) {
dir := t.TempDir()
target := filepath.Join(dir, "skill")
cacheDir := filepath.Join(target, "scripts", "rendering", "__pycache__")
if err := os.MkdirAll(cacheDir, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(target, "SKILL.md"), []byte("# Demo\n"), 0o644); err != nil {
t.Fatal(err)
}
bytecodePath := filepath.Join(cacheDir, "output.cpython-312.pyc")
if err := os.WriteFile(bytecodePath, []byte{0xcb, 0x0d, 0x0d, 0x0a, 0x00, 0x00, 0x00, 0x00}, 0o644); err != nil {
t.Fatal(err)
}
opts, err := ParseArgs([]string{target, "--scanner", "clawscan-static"})
if err != nil {
t.Fatal(err)
}
artifact, err := Run(opts, RunContext{Env: map[string]string{}})
if err != nil {
t.Fatal(err)
}
report := decodeStaticReport(t, artifact.Scanners["clawscan-static"].Raw)
wantPath := "scripts/rendering/__pycache__/output.cpython-312.pyc"
found := false
for _, finding := range report.Findings {
if finding.ID == "static.python_bytecode" && finding.Path == wantPath {
found = true
break
}
}
if !found {
t.Fatalf("missing Python bytecode finding: %#v", report.Findings)
}
prompt, err := RenderPromptTemplate("Evidence:\n{{ scanners.clawscan-static }}\nFiles:\n{{ target.files }}", artifact)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(prompt, `"id": "static.python_bytecode"`) {
t.Fatalf("judge prompt omitted static bytecode evidence: %s", prompt)
}
if !strings.Contains(prompt, wantPath+"\n[omitted: binary file]") {
t.Fatalf("judge prompt omitted bytecode path marker: %s", prompt)
}
if strings.Contains(prompt, "__pycache__\n[omitted: skipped path]") {
t.Fatalf("judge prompt still skipped __pycache__: %s", prompt)
}
}

func TestStaticScannerRecordsOmittedBinaryAndOversizedFiles(t *testing.T) {
dir := t.TempDir()
target := filepath.Join(dir, "skill")
Expand Down
39 changes: 31 additions & 8 deletions internal/runner/static_scanner.go
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,19 @@ var staticRules = []staticRule{
description: "Looks for destructive recursive removal of the filesystem root.",
pattern: regexp.MustCompile(`(?i)\brm\s+(?:-[a-z]*(?:r[a-z]*f|f[a-z]*r)[a-z]*|-[a-z]*r[a-z]*\s+-[a-z]*f[a-z]*|-[a-z]*f[a-z]*\s+-[a-z]*r[a-z]*)\s+/(?:\s|$)`),
},
{
id: "static.python_process_execution",
title: "Python process execution",
severity: "high",
description: "Looks for Python APIs that execute shell commands or child processes.",
pattern: regexp.MustCompile(`(?i)\b(?:os\s*\.\s*system|subprocess\s*\.\s*(?:call|check_call|check_output|popen|run))\s*\(`),
},
{
id: "static.python_bytecode",
title: "Packaged Python bytecode",
severity: "high",
description: "Flags precompiled Python bytecode that can execute without reviewable source.",
},
}

type staticFileCandidate struct {
Expand Down Expand Up @@ -182,6 +195,18 @@ func scanStaticTarget(target string) (staticScannerFiles, []staticFinding, error
files.addOmitted(rel, "not regular file", 0)
return nil
}
rel = filepath.ToSlash(rel)
if strings.EqualFold(filepath.Ext(rel), ".pyc") {
findings = append(findings, staticFinding{
ID: "static.python_bytecode",
Title: "Packaged Python bytecode",
Severity: "high",
Description: "Precompiled Python bytecode is executable and opaque to source-text review.",
Path: rel,
Line: 1,
Evidence: "Precompiled Python bytecode is included in the scanned target.",
})
}
if info.Size() > maxTargetFileBytes {
files.addOmitted(rel, "file exceeds size limit", info.Size())
return nil
Expand All @@ -200,7 +225,6 @@ func scanStaticTarget(target string) (staticScannerFiles, []staticFinding, error
return nil
}
totalBytes += info.Size()
rel = filepath.ToSlash(rel)
files.addScanned(rel, info.Size(), sha256BytesHex(content))
findings = append(findings, scanStaticContent(rel, string(content))...)
return nil
Expand All @@ -216,16 +240,12 @@ func scanStaticTarget(target string) (staticScannerFiles, []staticFinding, error
if err != nil {
return files.recordWalkError(target, path, entry)
}
if shouldSkipTargetPath(target, path) {
rel := relativeManifestPath(target, path)
if entry.IsDir() {
if entry.IsDir() {
if shouldSkipTargetDirectory(target, path) {
rel := relativeManifestPath(target, path)
files.addOmitted(rel, "skipped path", 0)
return filepath.SkipDir
}
files.addOmitted(rel, "skipped path", 0)
return nil
}
if entry.IsDir() {
return nil
}
info, err := entry.Info()
Expand Down Expand Up @@ -310,6 +330,9 @@ func scanStaticContent(path string, content string) []staticFinding {
lines := strings.Split(content, "\n")
for lineIndex, line := range lines {
for _, rule := range staticRules {
if rule.pattern == nil {
continue
}
if !rule.pattern.MatchString(line) {
continue
}
Expand Down
Loading