From d2c7e9336943499fbbe16ca8ae22bdc35a44e9e5 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 26 Jun 2026 13:02:03 -0300 Subject: [PATCH] docs: import more web skills + add their score->fix->re-score case studies MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Expanded the from-the-web run to 123 community skills across four public repos (added anthropics/skills). Every skill was safety-scanned as inert text first — 0/123 tripped the gate, none executed. New real before->after case studies (full skill dirs, verbatim, attributed): - daydream (glebis): no frontmatter at all, 38/F/Reject -> 100/A/Ship (+62) - disk-cleanup (glebis): invalid YAML frontmatter, 50/D -> 100/A (+50) - rehydrate (glebis): <> in description, 92/A -> 100/A (+8) Each "after" is the actual re-score of the fixed skill with zero remaining findings. Kept in docs/examples; from-the-web.md now leads with a four-case table. Synced the 69 -> 123 references on the benchmark README and landing lead. Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/examples/skill-benchmark/README.md | 13 +- docs/examples/skill-benchmark/from-the-web.md | 42 ++--- .../improved/web-daydream.after.json | 112 ++++++++++++++ .../improved/web-daydream.after.md | 14 ++ .../improved/web-daydream.before.json | 143 ++++++++++++++++++ .../improved/web-daydream.before.md | 20 +++ .../improved/web-daydream.compare.json | 11 ++ .../improved/web-disk-cleanup.after.json | 112 ++++++++++++++ .../improved/web-disk-cleanup.after.md | 14 ++ .../improved/web-disk-cleanup.before.json | 138 +++++++++++++++++ .../improved/web-disk-cleanup.before.md | 19 +++ .../improved/web-disk-cleanup.compare.json | 11 ++ .../improved/web-rehydrate.after.json | 112 ++++++++++++++ .../improved/web-rehydrate.after.md | 14 ++ .../improved/web-rehydrate.before.json | 118 +++++++++++++++ .../improved/web-rehydrate.before.md | 15 ++ .../improved/web-rehydrate.compare.json | 11 ++ docs/index.html | 2 +- 18 files changed, 896 insertions(+), 25 deletions(-) create mode 100644 docs/examples/skill-benchmark/improved/web-daydream.after.json create mode 100644 docs/examples/skill-benchmark/improved/web-daydream.after.md create mode 100644 docs/examples/skill-benchmark/improved/web-daydream.before.json create mode 100644 docs/examples/skill-benchmark/improved/web-daydream.before.md create mode 100644 docs/examples/skill-benchmark/improved/web-daydream.compare.json create mode 100644 docs/examples/skill-benchmark/improved/web-disk-cleanup.after.json create mode 100644 docs/examples/skill-benchmark/improved/web-disk-cleanup.after.md create mode 100644 docs/examples/skill-benchmark/improved/web-disk-cleanup.before.json create mode 100644 docs/examples/skill-benchmark/improved/web-disk-cleanup.before.md create mode 100644 docs/examples/skill-benchmark/improved/web-disk-cleanup.compare.json create mode 100644 docs/examples/skill-benchmark/improved/web-rehydrate.after.json create mode 100644 docs/examples/skill-benchmark/improved/web-rehydrate.after.md create mode 100644 docs/examples/skill-benchmark/improved/web-rehydrate.before.json create mode 100644 docs/examples/skill-benchmark/improved/web-rehydrate.before.md create mode 100644 docs/examples/skill-benchmark/improved/web-rehydrate.compare.json diff --git a/docs/examples/skill-benchmark/README.md b/docs/examples/skill-benchmark/README.md index 9442bc9..a5e1d08 100644 --- a/docs/examples/skill-benchmark/README.md +++ b/docs/examples/skill-benchmark/README.md @@ -52,13 +52,12 @@ clean**, the rest splitting into vendored-duplicate `SKILL.md` (e.g. [nextjs](ra ## From the web — real published skills -Beyond installed plugins, **[from-the-web.md](from-the-web.md)** scores **69 community -skills** pulled from three public GitHub collections (every one safety-scanned as inert -text first — 0 tripped the gate, none executed). It surfaced a real bug class — -**invalid YAML frontmatter** (an unquoted colon in `description`) — that used to *crash* -skval; skval now scores it instead, and the -[50 → 100 case study](from-the-web.md#case-study--invalid-frontmatter-50--100) shows the -one-line fix. +Beyond installed plugins, **[from-the-web.md](from-the-web.md)** scores **123 community +skills** pulled from four public GitHub collections (every one safety-scanned as inert +text first — 0 tripped the gate, none executed). It surfaced real, fixable classes — +missing/invalid frontmatter and `<>` in the description — and several +[score → fix → re-score case studies](from-the-web.md#case-studies--score--fix--re-score), +the biggest being **`daydream` 38 → 100** (no frontmatter → Ship). ## Case study 1 — `docx`: 92 → 100 (real skill) diff --git a/docs/examples/skill-benchmark/from-the-web.md b/docs/examples/skill-benchmark/from-the-web.md index b4d4073..ff5eaab 100644 --- a/docs/examples/skill-benchmark/from-the-web.md +++ b/docs/examples/skill-benchmark/from-the-web.md @@ -1,14 +1,14 @@ # From the web — skval over real published skills -A real `skval` **structural** run over **69 community skills** pulled from three public +A real `skval` **structural** run over **123 community skills** pulled from four public GitHub collections — [alirezarezvani/claude-skills](https://github.com/alirezarezvani/claude-skills), -[glebis/claude-skills](https://github.com/glebis/claude-skills), and -[smartnews/claude-skills](https://github.com/smartnews/claude-skills) — deterministic, no -model calls. Most are well-built; this page is about the ones skval flags, and what fixing -them looks like. +[glebis/claude-skills](https://github.com/glebis/claude-skills), +[smartnews/claude-skills](https://github.com/smartnews/claude-skills), and +[anthropics/skills](https://github.com/anthropics/skills) — deterministic, no model calls. +Most are well-built; this page is about the ones skval flags, and what fixing them looks like. > **Safety first.** Every fetched skill was treated as **inert text** and run through skval's -> D6 safety gate *before* anything else. **0 / 69** tripped it, and none were ever executed — +> D6 safety gate *before* anything else. **0 / 123** tripped it, and none were ever executed — > only the `SKILL.md` text was read and statically analysed. > > *(Symlink/pointer entries — where the raw file is just a path to another location — were @@ -33,20 +33,28 @@ finding and scores the skill. Pinned in [`tests/test_precision.py`](../../../tests/test_precision.py) (`test_invalid_yaml_frontmatter_scores_not_crashes`). -## Case study — `annotate`: 50 → 100 +## Case studies — score → fix → re-score -[`annotate`](https://github.com/glebis/claude-skills) (from glebis/claude-skills) — a real -published skill whose `description` has an unquoted colon, scored verbatim: +Real published skills, scored verbatim, then fixed against skval's ranked findings and +re-scored. Every "after" is the actual re-score of the fixed skill (no remaining findings). -| | Score | Grade | Verdict | Finding | -|---|------:|-------|---------|---------| -| **Before** ([scorecard](improved/web-frontmatter.before.md)) | 50 | D | Revise | invalid YAML frontmatter | -| **After** ([scorecard](improved/web-frontmatter.after.md)) | 100 | A | Ship | — | +| Skill | Source | Finding class | Before | After | Δ | +|-------|--------|---------------|:------:|:-----:|:--:| +| [`daydream`](https://github.com/glebis/claude-skills) | glebis | no frontmatter at all | [38 / F / Reject](improved/web-daydream.before.md) | [100 / A / Ship](improved/web-daydream.after.md) | **+62** | +| [`disk-cleanup`](https://github.com/glebis/claude-skills) | glebis | invalid YAML frontmatter | [50 / D / Revise](improved/web-disk-cleanup.before.md) | [100 / A / Ship](improved/web-disk-cleanup.after.md) | **+50** | +| [`annotate`](https://github.com/glebis/claude-skills) | glebis | invalid YAML frontmatter | [50 / D / Revise](improved/web-frontmatter.before.md) | [100 / A / Ship](improved/web-frontmatter.after.md) | **+50** | +| [`rehydrate`](https://github.com/glebis/claude-skills) | glebis | `<` / `>` in description | [92 / A / Ship](improved/web-rehydrate.before.md) | [100 / A / Ship](improved/web-rehydrate.after.md) | **+8** | -The fix skval points to — **quote the description** — is one line. skval's own diff -([`improved/web-frontmatter.compare.json`](improved/web-frontmatter.compare.json)): -`overall_delta: +50`, verdict `Revise → Ship`. What changed: -[`improved/web-frontmatter.change.md`](improved/web-frontmatter.change.md). +**The fixes are tiny, one each:** + +- **`daydream`** had *no YAML frontmatter* — no `name`, no `description`, so it can't trigger. + Adding a frontmatter block takes it from **38 / F / Reject → 100 / A / Ship** (skval's biggest + turnaround in the set, `overall_delta +62`). What changed: + [`web-frontmatter.change.md`](improved/web-frontmatter.change.md) shows the same class on `annotate`. +- **`disk-cleanup` / `annotate`** ship an unquoted colon in `description` (`… Local-only: synthetic …`) + that breaks the YAML — quote the description and both go **50 → 100**. +- **`rehydrate`** has `<` / `>` in `description` (breaks parsers / triggering) — drop the angle + brackets, **92 → 100**. --- diff --git a/docs/examples/skill-benchmark/improved/web-daydream.after.json b/docs/examples/skill-benchmark/improved/web-daydream.after.json new file mode 100644 index 0000000..88fb0db --- /dev/null +++ b/docs/examples/skill-benchmark/improved/web-daydream.after.json @@ -0,0 +1,112 @@ +{ + "score": 100, + "grade": "A", + "verdict": "Ship", + "safety_pass": true, + "dimensions": { + "D1": { + "score": 1.0, + "weight": 0.15, + "checks": [ + { + "id": "frontmatter_present", + "passed": true, + "severity": "critical", + "detail": "" + }, + { + "id": "frontmatter_valid_yaml", + "passed": true, + "severity": "critical", + "detail": "" + }, + { + "id": "name_present", + "passed": true, + "severity": "critical", + "detail": "" + }, + { + "id": "name_kebab_case", + "passed": true, + "severity": "major", + "detail": "" + }, + { + "id": "description_present", + "passed": true, + "severity": "critical", + "detail": "" + }, + { + "id": "description_len", + "passed": true, + "severity": "major", + "detail": "" + }, + { + "id": "description_no_angle_brackets", + "passed": true, + "severity": "major", + "detail": "" + }, + { + "id": "allowed_frontmatter_keys", + "passed": true, + "severity": "minor", + "detail": "" + }, + { + "id": "single_skill_md", + "passed": true, + "severity": "critical", + "detail": "" + }, + { + "id": "skill_md_line_budget", + "passed": true, + "severity": "minor", + "detail": "" + }, + { + "id": "token_budget", + "passed": true, + "severity": "minor", + "detail": "" + }, + { + "id": "no_broken_local_refs", + "passed": true, + "severity": "major", + "detail": "" + } + ] + } + }, + "safety": { + "safety_pass": true, + "findings": [] + }, + "findings": [], + "provenance": { + "source": "github.com/glebis/claude-skills \u00b7 daydream", + "kind": "dir", + "resolved_path": "github.com/glebis/claude-skills \u00b7 daydream" + }, + "metadata": { + "mode": "structural-only", + "skill_name": "daydream", + "classification": { + "type": "task", + "confidence": "medium", + "also": [] + }, + "timestamp": "2026-06-26T15:59:31Z", + "skipped_dimensions": [ + "D2", + "D3", + "D4", + "D5" + ] + } +} \ No newline at end of file diff --git a/docs/examples/skill-benchmark/improved/web-daydream.after.md b/docs/examples/skill-benchmark/improved/web-daydream.after.md new file mode 100644 index 0000000..9de7c0e --- /dev/null +++ b/docs/examples/skill-benchmark/improved/web-daydream.after.md @@ -0,0 +1,14 @@ +# Skill Scorecard — `daydream` +Source: github.com/glebis/claude-skills · daydream · mode: structural-only +Type: task (confidence: medium) + +## ██████████████ 100 / 100 Grade: A Verdict: Ship + +| Dimension | Score | Weight | Detail | +|-----------|-------|--------|--------| +| D1 Structural | 1.00 | 0.15 | all checks pass | + +Safety gate: PASS +Skipped dimensions (structural-only): D2, D3, D4, D5 + +No findings — clean. diff --git a/docs/examples/skill-benchmark/improved/web-daydream.before.json b/docs/examples/skill-benchmark/improved/web-daydream.before.json new file mode 100644 index 0000000..c745fe3 --- /dev/null +++ b/docs/examples/skill-benchmark/improved/web-daydream.before.json @@ -0,0 +1,143 @@ +{ + "score": 38, + "grade": "F", + "verdict": "Reject", + "safety_pass": true, + "dimensions": { + "D1": { + "score": 0.3846, + "weight": 0.15, + "checks": [ + { + "id": "frontmatter_present", + "passed": false, + "severity": "critical", + "detail": "missing leading '---'" + }, + { + "id": "frontmatter_valid_yaml", + "passed": false, + "severity": "critical", + "detail": "no YAML frontmatter found" + }, + { + "id": "name_present", + "passed": false, + "severity": "critical", + "detail": "missing 'name'" + }, + { + "id": "name_kebab_case", + "passed": false, + "severity": "major", + "detail": "name '' must be kebab-case, <=64 chars" + }, + { + "id": "description_present", + "passed": false, + "severity": "critical", + "detail": "missing 'description'" + }, + { + "id": "description_len", + "passed": false, + "severity": "major", + "detail": "description is 0 chars (max 1024)" + }, + { + "id": "description_no_angle_brackets", + "passed": true, + "severity": "major", + "detail": "" + }, + { + "id": "allowed_frontmatter_keys", + "passed": true, + "severity": "minor", + "detail": "" + }, + { + "id": "single_skill_md", + "passed": true, + "severity": "critical", + "detail": "" + }, + { + "id": "skill_md_line_budget", + "passed": true, + "severity": "minor", + "detail": "" + }, + { + "id": "token_budget", + "passed": true, + "severity": "minor", + "detail": "" + }, + { + "id": "no_broken_local_refs", + "passed": true, + "severity": "major", + "detail": "" + } + ] + } + }, + "safety": { + "safety_pass": true, + "findings": [] + }, + "findings": [ + { + "dimension": "D1", + "impact_estimate": 3, + "message": "missing leading '---'" + }, + { + "dimension": "D1", + "impact_estimate": 3, + "message": "no YAML frontmatter found" + }, + { + "dimension": "D1", + "impact_estimate": 3, + "message": "missing 'name'" + }, + { + "dimension": "D1", + "impact_estimate": 3, + "message": "missing 'description'" + }, + { + "dimension": "D1", + "impact_estimate": 2, + "message": "name '' must be kebab-case, <=64 chars" + }, + { + "dimension": "D1", + "impact_estimate": 2, + "message": "description is 0 chars (max 1024)" + } + ], + "provenance": { + "source": "github.com/glebis/claude-skills \u00b7 daydream", + "kind": "dir", + "resolved_path": "github.com/glebis/claude-skills \u00b7 daydream" + }, + "metadata": { + "mode": "structural-only", + "skill_name": null, + "classification": { + "type": "task", + "confidence": "medium", + "also": [] + }, + "timestamp": "2026-06-26T15:59:31Z", + "skipped_dimensions": [ + "D2", + "D3", + "D4", + "D5" + ] + } +} \ No newline at end of file diff --git a/docs/examples/skill-benchmark/improved/web-daydream.before.md b/docs/examples/skill-benchmark/improved/web-daydream.before.md new file mode 100644 index 0000000..f806087 --- /dev/null +++ b/docs/examples/skill-benchmark/improved/web-daydream.before.md @@ -0,0 +1,20 @@ +# Skill Scorecard — `claude-skills · daydream` +Source: github.com/glebis/claude-skills · daydream · mode: structural-only +Type: task (confidence: medium) + +## █████░░░░░░░░░ 38 / 100 Grade: F Verdict: Reject + +| Dimension | Score | Weight | Detail | +|-----------|-------|--------|--------| +| D1 Structural | 0.38 | 0.15 | 6/12 checks failed | + +Safety gate: PASS +Skipped dimensions (structural-only): D2, D3, D4, D5 + +## Top findings (ranked by score impact) +1. [D1] missing leading '---' +2. [D1] no YAML frontmatter found +3. [D1] missing 'name' +4. [D1] missing 'description' +5. [D1] name '' must be kebab-case, <=64 chars +6. [D1] description is 0 chars (max 1024) diff --git a/docs/examples/skill-benchmark/improved/web-daydream.compare.json b/docs/examples/skill-benchmark/improved/web-daydream.compare.json new file mode 100644 index 0000000..ef6e039 --- /dev/null +++ b/docs/examples/skill-benchmark/improved/web-daydream.compare.json @@ -0,0 +1,11 @@ +{ + "overall_delta": 62, + "dimensions": { + "D1": { + "a": 0.3846, + "b": 1.0, + "delta": 0.6154 + } + }, + "verdict_change": "Reject -> Ship" +} \ No newline at end of file diff --git a/docs/examples/skill-benchmark/improved/web-disk-cleanup.after.json b/docs/examples/skill-benchmark/improved/web-disk-cleanup.after.json new file mode 100644 index 0000000..a002fcd --- /dev/null +++ b/docs/examples/skill-benchmark/improved/web-disk-cleanup.after.json @@ -0,0 +1,112 @@ +{ + "score": 100, + "grade": "A", + "verdict": "Ship", + "safety_pass": true, + "dimensions": { + "D1": { + "score": 1.0, + "weight": 0.15, + "checks": [ + { + "id": "frontmatter_present", + "passed": true, + "severity": "critical", + "detail": "" + }, + { + "id": "frontmatter_valid_yaml", + "passed": true, + "severity": "critical", + "detail": "" + }, + { + "id": "name_present", + "passed": true, + "severity": "critical", + "detail": "" + }, + { + "id": "name_kebab_case", + "passed": true, + "severity": "major", + "detail": "" + }, + { + "id": "description_present", + "passed": true, + "severity": "critical", + "detail": "" + }, + { + "id": "description_len", + "passed": true, + "severity": "major", + "detail": "" + }, + { + "id": "description_no_angle_brackets", + "passed": true, + "severity": "major", + "detail": "" + }, + { + "id": "allowed_frontmatter_keys", + "passed": true, + "severity": "minor", + "detail": "" + }, + { + "id": "single_skill_md", + "passed": true, + "severity": "critical", + "detail": "" + }, + { + "id": "skill_md_line_budget", + "passed": true, + "severity": "minor", + "detail": "" + }, + { + "id": "token_budget", + "passed": true, + "severity": "minor", + "detail": "" + }, + { + "id": "no_broken_local_refs", + "passed": true, + "severity": "major", + "detail": "" + } + ] + } + }, + "safety": { + "safety_pass": true, + "findings": [] + }, + "findings": [], + "provenance": { + "source": "github.com/glebis/claude-skills \u00b7 disk-cleanup", + "kind": "dir", + "resolved_path": "github.com/glebis/claude-skills \u00b7 disk-cleanup" + }, + "metadata": { + "mode": "structural-only", + "skill_name": "disk-cleanup", + "classification": { + "type": "task", + "confidence": "medium", + "also": [] + }, + "timestamp": "2026-06-26T15:59:33Z", + "skipped_dimensions": [ + "D2", + "D3", + "D4", + "D5" + ] + } +} \ No newline at end of file diff --git a/docs/examples/skill-benchmark/improved/web-disk-cleanup.after.md b/docs/examples/skill-benchmark/improved/web-disk-cleanup.after.md new file mode 100644 index 0000000..cf1b95e --- /dev/null +++ b/docs/examples/skill-benchmark/improved/web-disk-cleanup.after.md @@ -0,0 +1,14 @@ +# Skill Scorecard — `disk-cleanup` +Source: github.com/glebis/claude-skills · disk-cleanup · mode: structural-only +Type: task (confidence: medium) + +## ██████████████ 100 / 100 Grade: A Verdict: Ship + +| Dimension | Score | Weight | Detail | +|-----------|-------|--------|--------| +| D1 Structural | 1.00 | 0.15 | all checks pass | + +Safety gate: PASS +Skipped dimensions (structural-only): D2, D3, D4, D5 + +No findings — clean. diff --git a/docs/examples/skill-benchmark/improved/web-disk-cleanup.before.json b/docs/examples/skill-benchmark/improved/web-disk-cleanup.before.json new file mode 100644 index 0000000..e0926ee --- /dev/null +++ b/docs/examples/skill-benchmark/improved/web-disk-cleanup.before.json @@ -0,0 +1,138 @@ +{ + "score": 50, + "grade": "D", + "verdict": "Revise", + "safety_pass": true, + "dimensions": { + "D1": { + "score": 0.5, + "weight": 0.15, + "checks": [ + { + "id": "frontmatter_present", + "passed": true, + "severity": "critical", + "detail": "" + }, + { + "id": "frontmatter_valid_yaml", + "passed": false, + "severity": "critical", + "detail": "invalid YAML frontmatter: mapping values are not allowed here" + }, + { + "id": "name_present", + "passed": false, + "severity": "critical", + "detail": "missing 'name'" + }, + { + "id": "name_kebab_case", + "passed": false, + "severity": "major", + "detail": "name '' must be kebab-case, <=64 chars" + }, + { + "id": "description_present", + "passed": false, + "severity": "critical", + "detail": "missing 'description'" + }, + { + "id": "description_len", + "passed": false, + "severity": "major", + "detail": "description is 0 chars (max 1024)" + }, + { + "id": "description_no_angle_brackets", + "passed": true, + "severity": "major", + "detail": "" + }, + { + "id": "allowed_frontmatter_keys", + "passed": true, + "severity": "minor", + "detail": "" + }, + { + "id": "single_skill_md", + "passed": true, + "severity": "critical", + "detail": "" + }, + { + "id": "skill_md_line_budget", + "passed": true, + "severity": "minor", + "detail": "" + }, + { + "id": "token_budget", + "passed": true, + "severity": "minor", + "detail": "" + }, + { + "id": "no_broken_local_refs", + "passed": true, + "severity": "major", + "detail": "" + } + ] + } + }, + "safety": { + "safety_pass": true, + "findings": [] + }, + "findings": [ + { + "dimension": "D1", + "impact_estimate": 3, + "message": "invalid YAML frontmatter: mapping values are not allowed here" + }, + { + "dimension": "D1", + "impact_estimate": 3, + "message": "missing 'name'" + }, + { + "dimension": "D1", + "impact_estimate": 3, + "message": "missing 'description'" + }, + { + "dimension": "D1", + "impact_estimate": 2, + "message": "name '' must be kebab-case, <=64 chars" + }, + { + "dimension": "D1", + "impact_estimate": 2, + "message": "description is 0 chars (max 1024)" + } + ], + "provenance": { + "source": "github.com/glebis/claude-skills \u00b7 disk-cleanup", + "kind": "dir", + "resolved_path": "github.com/glebis/claude-skills \u00b7 disk-cleanup" + }, + "metadata": { + "mode": "structural-only", + "skill_name": null, + "classification": { + "type": "task", + "confidence": "medium", + "also": [] + }, + "timestamp": "2026-06-26T15:59:33Z", + "skipped_dimensions": [ + "D2", + "D3", + "D4", + "D5" + ] + } +} \ No newline at end of file diff --git a/docs/examples/skill-benchmark/improved/web-disk-cleanup.before.md b/docs/examples/skill-benchmark/improved/web-disk-cleanup.before.md new file mode 100644 index 0000000..2abce0d --- /dev/null +++ b/docs/examples/skill-benchmark/improved/web-disk-cleanup.before.md @@ -0,0 +1,19 @@ +# Skill Scorecard — `claude-skills · disk-cleanup` +Source: github.com/glebis/claude-skills · disk-cleanup · mode: structural-only +Type: task (confidence: medium) + +## ███████░░░░░░░ 50 / 100 Grade: D Verdict: Revise + +| Dimension | Score | Weight | Detail | +|-----------|-------|--------|--------| +| D1 Structural | 0.50 | 0.15 | 5/12 checks failed | + +Safety gate: PASS +Skipped dimensions (structural-only): D2, D3, D4, D5 + +## Top findings (ranked by score impact) +1. [D1] invalid YAML frontmatter: mapping values are not allowed here +2. [D1] missing 'name' +3. [D1] missing 'description' +4. [D1] name '' must be kebab-case, <=64 chars +5. [D1] description is 0 chars (max 1024) diff --git a/docs/examples/skill-benchmark/improved/web-disk-cleanup.compare.json b/docs/examples/skill-benchmark/improved/web-disk-cleanup.compare.json new file mode 100644 index 0000000..409f85c --- /dev/null +++ b/docs/examples/skill-benchmark/improved/web-disk-cleanup.compare.json @@ -0,0 +1,11 @@ +{ + "overall_delta": 50, + "dimensions": { + "D1": { + "a": 0.5, + "b": 1.0, + "delta": 0.5 + } + }, + "verdict_change": "Revise -> Ship" +} \ No newline at end of file diff --git a/docs/examples/skill-benchmark/improved/web-rehydrate.after.json b/docs/examples/skill-benchmark/improved/web-rehydrate.after.json new file mode 100644 index 0000000..5af30f5 --- /dev/null +++ b/docs/examples/skill-benchmark/improved/web-rehydrate.after.json @@ -0,0 +1,112 @@ +{ + "score": 100, + "grade": "A", + "verdict": "Ship", + "safety_pass": true, + "dimensions": { + "D1": { + "score": 1.0, + "weight": 0.15, + "checks": [ + { + "id": "frontmatter_present", + "passed": true, + "severity": "critical", + "detail": "" + }, + { + "id": "frontmatter_valid_yaml", + "passed": true, + "severity": "critical", + "detail": "" + }, + { + "id": "name_present", + "passed": true, + "severity": "critical", + "detail": "" + }, + { + "id": "name_kebab_case", + "passed": true, + "severity": "major", + "detail": "" + }, + { + "id": "description_present", + "passed": true, + "severity": "critical", + "detail": "" + }, + { + "id": "description_len", + "passed": true, + "severity": "major", + "detail": "" + }, + { + "id": "description_no_angle_brackets", + "passed": true, + "severity": "major", + "detail": "" + }, + { + "id": "allowed_frontmatter_keys", + "passed": true, + "severity": "minor", + "detail": "" + }, + { + "id": "single_skill_md", + "passed": true, + "severity": "critical", + "detail": "" + }, + { + "id": "skill_md_line_budget", + "passed": true, + "severity": "minor", + "detail": "" + }, + { + "id": "token_budget", + "passed": true, + "severity": "minor", + "detail": "" + }, + { + "id": "no_broken_local_refs", + "passed": true, + "severity": "major", + "detail": "" + } + ] + } + }, + "safety": { + "safety_pass": true, + "findings": [] + }, + "findings": [], + "provenance": { + "source": "github.com/glebis/claude-skills \u00b7 rehydrate", + "kind": "dir", + "resolved_path": "github.com/glebis/claude-skills \u00b7 rehydrate" + }, + "metadata": { + "mode": "structural-only", + "skill_name": "rehydrate", + "classification": { + "type": "task", + "confidence": "medium", + "also": [] + }, + "timestamp": "2026-06-26T15:59:35Z", + "skipped_dimensions": [ + "D2", + "D3", + "D4", + "D5" + ] + } +} \ No newline at end of file diff --git a/docs/examples/skill-benchmark/improved/web-rehydrate.after.md b/docs/examples/skill-benchmark/improved/web-rehydrate.after.md new file mode 100644 index 0000000..c0c9e10 --- /dev/null +++ b/docs/examples/skill-benchmark/improved/web-rehydrate.after.md @@ -0,0 +1,14 @@ +# Skill Scorecard — `rehydrate` +Source: github.com/glebis/claude-skills · rehydrate · mode: structural-only +Type: task (confidence: medium) + +## ██████████████ 100 / 100 Grade: A Verdict: Ship + +| Dimension | Score | Weight | Detail | +|-----------|-------|--------|--------| +| D1 Structural | 1.00 | 0.15 | all checks pass | + +Safety gate: PASS +Skipped dimensions (structural-only): D2, D3, D4, D5 + +No findings — clean. diff --git a/docs/examples/skill-benchmark/improved/web-rehydrate.before.json b/docs/examples/skill-benchmark/improved/web-rehydrate.before.json new file mode 100644 index 0000000..82a714d --- /dev/null +++ b/docs/examples/skill-benchmark/improved/web-rehydrate.before.json @@ -0,0 +1,118 @@ +{ + "score": 92, + "grade": "A", + "verdict": "Ship", + "safety_pass": true, + "dimensions": { + "D1": { + "score": 0.9231, + "weight": 0.15, + "checks": [ + { + "id": "frontmatter_present", + "passed": true, + "severity": "critical", + "detail": "" + }, + { + "id": "frontmatter_valid_yaml", + "passed": true, + "severity": "critical", + "detail": "" + }, + { + "id": "name_present", + "passed": true, + "severity": "critical", + "detail": "" + }, + { + "id": "name_kebab_case", + "passed": true, + "severity": "major", + "detail": "" + }, + { + "id": "description_present", + "passed": true, + "severity": "critical", + "detail": "" + }, + { + "id": "description_len", + "passed": true, + "severity": "major", + "detail": "" + }, + { + "id": "description_no_angle_brackets", + "passed": false, + "severity": "major", + "detail": "description contains '<' or '>'" + }, + { + "id": "allowed_frontmatter_keys", + "passed": true, + "severity": "minor", + "detail": "" + }, + { + "id": "single_skill_md", + "passed": true, + "severity": "critical", + "detail": "" + }, + { + "id": "skill_md_line_budget", + "passed": true, + "severity": "minor", + "detail": "" + }, + { + "id": "token_budget", + "passed": true, + "severity": "minor", + "detail": "" + }, + { + "id": "no_broken_local_refs", + "passed": true, + "severity": "major", + "detail": "" + } + ] + } + }, + "safety": { + "safety_pass": true, + "findings": [] + }, + "findings": [ + { + "dimension": "D1", + "impact_estimate": 2, + "message": "description contains '<' or '>'" + } + ], + "provenance": { + "source": "github.com/glebis/claude-skills \u00b7 rehydrate", + "kind": "dir", + "resolved_path": "github.com/glebis/claude-skills \u00b7 rehydrate" + }, + "metadata": { + "mode": "structural-only", + "skill_name": "rehydrate", + "classification": { + "type": "task", + "confidence": "medium", + "also": [] + }, + "timestamp": "2026-06-26T15:59:35Z", + "skipped_dimensions": [ + "D2", + "D3", + "D4", + "D5" + ] + } +} \ No newline at end of file diff --git a/docs/examples/skill-benchmark/improved/web-rehydrate.before.md b/docs/examples/skill-benchmark/improved/web-rehydrate.before.md new file mode 100644 index 0000000..f5b89ce --- /dev/null +++ b/docs/examples/skill-benchmark/improved/web-rehydrate.before.md @@ -0,0 +1,15 @@ +# Skill Scorecard — `rehydrate` +Source: github.com/glebis/claude-skills · rehydrate · mode: structural-only +Type: task (confidence: medium) + +## █████████████░ 92 / 100 Grade: A Verdict: Ship + +| Dimension | Score | Weight | Detail | +|-----------|-------|--------|--------| +| D1 Structural | 0.92 | 0.15 | 1/12 checks failed | + +Safety gate: PASS +Skipped dimensions (structural-only): D2, D3, D4, D5 + +## Top findings (ranked by score impact) +1. [D1] description contains '<' or '>' diff --git a/docs/examples/skill-benchmark/improved/web-rehydrate.compare.json b/docs/examples/skill-benchmark/improved/web-rehydrate.compare.json new file mode 100644 index 0000000..96aeb51 --- /dev/null +++ b/docs/examples/skill-benchmark/improved/web-rehydrate.compare.json @@ -0,0 +1,11 @@ +{ + "overall_delta": 8, + "dimensions": { + "D1": { + "a": 0.9231, + "b": 1.0, + "delta": 0.0769 + } + }, + "verdict_change": "Ship" +} \ No newline at end of file diff --git a/docs/index.html b/docs/index.html index 53104c5..897ddfa 100644 --- a/docs/index.html +++ b/docs/index.html @@ -172,7 +172,7 @@

skval vs. skill-creator

Benchmark

skval's deterministic structural scan over 10 widely-used skills — 8 ship as-is; the two flagged hit the size budget. Plus runs over 75 installed skills - and 69 from the web. + and 123 from the web. See the full benchmark →

SkillTypeScoreFindings