An agent skill for auditing web discoverability, built around a machine-readable requirement registry, a profile-driven applicability engine, deterministic checks that read served output, and a benchmark harness that scores the checks against fixture projects with planted defects.
It is designed to be used by a coding agent — Claude Code, Codex, Cursor, Copilot Workspace, Antigravity — and by people, from the command line.
Its purpose is to answer six questions about any recommendation it makes:
| Question | Where the answer comes from |
|---|---|
| Why did you recommend this? | the requirement's statement and rationale |
| What evidence supports it? | its sources[], each with an evidence tier |
| Does this apply to this site? | the detected profile and the requirement's applies_when |
| What changed? | the finding's location and its captured evidence |
| How do we know the change worked? | verification.level and verification.method |
| How confident are we? | evidence_tier and confidence, which are independent |
It does not improve search rankings, and nothing in this repository is evidence that it could. It does not cause a page to be indexed, or cited by an AI answer engine. Crawling, indexing, ranking, and citation are decided by platforms whose behaviour this repository cannot observe and does not model.
What it does is narrower and checkable: it identifies places where a site's served output contradicts documented, published expectations of search engines and crawlers, cites the document it is relying on, and states what would have to be observed for the problem to be considered fixed.
Read LIMITATIONS.md before relying on any of it. It is not a disclaimer appendix; it is a list of things this repository genuinely cannot tell you.
| Registry | Value |
|---|---|
| Active requirements | 224 |
| Distinct titles | 224 |
| With a citation | 204 |
| With a verification method | 224 |
| With a deterministic check | 36 (16%) |
| With a confidence rating | 224 |
| Duplicate titles | 0 |
| Near-duplicate title pairs | 0 |
| Broken dependencies | 0 |
| Circular dependencies | 0 |
| Retired IDs (never reused) | 80 |
| Deferred IDs | 336 |
| Deterministic checks | 36 |
| Benchmark (fixtures, checked subset only) | Value |
|---|---|
| Benchmark cases | 12/12 passing |
| Precision | 1 |
| Recall | 1 |
| False-positive rate | 0 |
| False-negative rate | 0 |
| Verification accuracy | 1 |
Benchmarks measure the checked subset of the registry against fixture projects with planted defects. They do not measure search rankings or AI answer inclusion, and are not evidence of either. See BENCHMARKS.md.
Regenerate with npm run bench && npm run docs -- --write. No number in that
table is typed by hand, and npm run ci fails if any of them drifts.
The bundled audit tooling supports Node.js 22.18 or newer and has no dependencies
or build step: the tools are TypeScript executed directly by Node's native type
stripping, so there is nothing to compile and nothing to install. Vercel's
skills CLI has its own Node requirement; its current release requires Node.js
22.20 or newer.
This repository is compatible with Vercel's open skills CLI. The root
SKILL.md is the canonical skill entry, so install it directly from GitHub:
npx skills add ARafaykhalid/web-discoverability-skill \
--skill web-discoverability-skillThe CLI detects installed agents automatically. To install globally for Codex without prompts, use:
npx skills add ARafaykhalid/web-discoverability-skill \
--skill web-discoverability-skill --global --agent codex --yesTo use the skill for one task without installing it:
npx skills use ARafaykhalid/web-discoverability-skill@web-discoverability-skill \
--agent codexThe equivalent full URL is
https://github.com/ARafaykhalid/web-discoverability-skill.
git clone https://github.com/ARafaykhalid/web-discoverability-skill
cd web-discoverability-skill
npm testThe skill install and the CLI are separate concerns. The Vercel skills command
installs the instructions and bundled references for an agent; the repository
itself declares no dependencies or devDependencies, and every tool is
TypeScript run directly by node (no compiler, no transpilation step, no
artifacts). npm link puts the wds command on your path if you want the
audit tooling outside this directory.
node tools/cli.ts version # skill, registry, and upstream freshness
node tools/cli.ts profile ./path/to/site # what kind of site is this?
node tools/cli.ts select ./path/to/site # which rules apply, and why
node tools/cli.ts audit ./path/to/site # run the checks, report findingsversion is the freshness check the skill runs before any substantive work. It
reports the skill version, the registry schema version, the commit the local
copy is at, and the latest commit on upstream main; when the network is
unavailable it says unable_to_verify with the reason rather than pretending
the copy is current. It always exits 0, because "could not check" is a recorded
outcome, not a failure. audit, select, and profile embed skill_version
and registry_version in their JSON output without touching the network.
profile, select, and audit never modify the project they inspect. With
--write they may persist reports under reports/. Registry-maintenance
commands are separate: docs --write regenerates derived documentation and
schemas, while the bench and metrics npm scripts refresh their reports.
Registry maintenance and self-checks:
npm run version # skill, registry, and upstream freshness check
npm run validate # registry integrity; exits 1 on any error
npm run quality # duplicates, dependencies, missing metadata
npm run metrics # the machine-readable metrics report
npm run stale # requirements overdue for source re-verification
npm run sources # the source manifest, one row per citation
npm run check-sources # live reachability of every cited URL
npm run bench # score the fixture cases, record the result
npm run docs:check # fail if any generated artifact has drifted
npm run ci # all of the above that do not need the networkTo regenerate derived Markdown, schemas, and indexes after editing canonical
records, run npm run docs -- --write. Plain npm run docs is read-only.
Audit. Implemented in this repository, as tools/cli.ts audit. Detects the
profile, selects applicable requirements, runs the deterministic checks against a
snapshot of served output, and reports findings with evidence and locations. This
is the mode with test coverage and benchmark scores.
Verification. Also implemented, and not separable from audit: every check
declares whether it reads source files, runtime output, or both, and a check that
needs runtime output it does not have reports NEEDS_RUNTIME rather than
guessing. A source file containing a <meta> tag is not accepted as evidence
that the served document contains it.
Implementation. Performed by the agent, following SKILL.md, not
by the tooling. There is no apply or fix command, and adding one is not on
the near-term list. What the registry contributes is the change_safety
classification that governs what an agent may change without asking:
change_safety |
Meaning |
|---|---|
SAFE_AUTOMATIC |
deterministic, reversible, and cannot affect indexing directives, URLs, or business facts; an agent may just do it |
REVIEW_REQUIRED |
could affect content, URLs, indexing, redirects, canonicalisation, structured data, or application behaviour; propose a diff and wait for approval |
BLOCKED |
needs credentials, external platform access, a business decision, or facts the repository does not contain; report it, never implement it |
A requirement whose impact is HIGH can never be SAFE_AUTOMATIC; the
validator rejects that combination. The existence of a requirement is not a
reason to change a site: see AUDIT.md for the intended workflow, the
per-requirement outcome vocabulary, and the safety model in full.
Every requirement that makes an externally checkable claim cites at least one source, and each citation carries a tier describing how strongly that source supports this claim:
| Tier | Meaning |
|---|---|
| A | official specification or platform documentation — Google Search Central, Bing, schema.org, an RFC, a framework's own docs, an official crawler reference |
| B | strong technical consensus among practitioners, without an official document saying it |
| C | empirical or observational — someone measured it, and the measurement is public |
| D | experimental or speculative; a hypothesis, not a finding |
INTERNAL |
a rule about this repository's own conventions, which needs no external source |
A requirement's own evidence_tier must appear among the tiers of its citations,
so a record cannot claim tier A support while citing only tier C material. Tier
D records may not carry HIGH confidence and must carry an explicit caution.
The tier describes the support, not the document's prestige: the same URL can
legitimately be tier A for a claim it states outright and tier B for a claim it
merely implies.
Confidence is a separate axis. A tier A source can support a rule we are nonetheless unsure how to apply, and a tier C observation can be one we are confident about.
The full manifest, with the date each citation was last checked, is
EVIDENCE.md and reports/sources.json.
The registry keeps these apart on purpose, because they are not equally supported:
| Category | What it covers |
|---|---|
TECHNICAL_SEO |
crawlability, canonicalisation, URLs, status codes, sitemaps, robots |
SEARCH_DISCOVERABILITY |
metadata and signals search engines document reading |
STRUCTURED_DATA_ELIGIBILITY |
markup that makes a page eligible for a rich result |
CONTENT_QUALITY |
clarity, accuracy, attribution, and dates |
ACCESSIBILITY |
accessibility work that also affects machine readability |
SECURITY_PRIVACY |
keeping private things private and unindexed |
OPERATIONS |
monitoring, review intervals, and change management |
AI_CRAWLER_ACCESS |
documented behaviour of named AI crawlers |
AI_RETRIEVAL |
making content parseable by retrieval systems |
EMERGING_GEO |
experimental generative-engine practice, labelled as such |
Only EMERGING_GEO is treated as experimental by the validator. AI_CRAWLER_ACCESS
is not, because crawler user-agent strings and their documented directives are
published facts, and a rule about them can cite a tier A source.
Specifically, this repository does not claim that publishing llms.txt makes a
site visible to an AI assistant, that allowing a crawler causes inclusion, that
structured data affects ranking, or that semantic HTML produces citations. Where
a practice is plausible but unconfirmed, the record says so and carries a tier D
citation and a caution.
Support is uneven, and the table says how. "Fixture" means a benchmark project exists and the checks are scored against it; "adapter" means there is framework-specific implementation guidance in references/framework-adapters.md; "detected" means the profiler recognises the framework and applies generic guidance.
| Stack | Status |
|---|---|
| Static HTML | fixture, detected |
| Vite (prerendered and client-rendered) | fixture, detected |
| React SPA | fixture, detected |
| Next.js (App Router and Pages Router) | adapter, detected |
| Astro | adapter, detected |
| Nuxt, SvelteKit, Remix, Gatsby | adapter, detected |
| Express and other plain Node servers | adapter, detected |
| Django, Flask, FastAPI, Rails, Laravel | detected only |
| Anything else | generic guidance; the profiler reports unknown and says so |
Site types the applicability model distinguishes — ecommerce, SaaS, editorial and
news, documentation, local business, multilingual, user-generated content,
paywall and subscription — gate requirements on detected facts rather than on a
declared site category. Where the profiler cannot tell, it reports the fact as
unknown and the requirement as UNCERTAIN, which means ask for evidence, not
change the file anyway.
Which of those site types has a benchmark fixture is recorded in benchmarks/README.md, including the ones that do not.
requirements/ canonical JSONL file per domain, plus removed.jsonl and deferred.jsonl
manifest.json generated index: domain counts, level candidates
registry.md generated human-readable index
by-id/ generated Markdown page for every active requirement
schema/ generated JSON Schema for records, ledgers, and benchmark cases
tools/
cli.ts the wds command; every subcommand lives here
lib/ registry, validator, profiler, selector, reporter, version, docs
lib/checks/ the deterministic checks, one module per domain
benchmarks/
cases/ one JSON case per fixture: declared defects and expectations
fixtures/ the fixture projects themselves
tests/ node:test suites: registry, tools, checks, regression
references/ deep reference documents the agent reads on demand
assets/templates/ report templates
agents/ agent-framework definitions
reports/ generated; not authoritative, and safe to delete
SKILL.md the agent orchestrator instructions
Generated files carry a header saying so. Editing one is pointless — npm run docs -- --write overwrites it, and npm run ci fails while it disagrees with
the registry.
- Findings are not permission. A finding says a requirement is unsatisfied.
Whether to change anything is governed by
change_safety, byimpact, and by the person running the tool. - Uncertainty blocks, it does not default. An unknown profile fact yields
UNCERTAIN, neverNOT_APPLICABLEand never a change. The agent is required to ask for evidence. - Nothing is fabricated. Reviews, ratings, aggregate ratings, authorship,
organisation details, and dates must come from real data. The requirements
covering them are
BLOCKED, which means an agent may report them and may not act on them. - Verification reads output, not intent. Checks in runtime-sensitive domains may not be satisfied by source inspection alone.
- High impact means slow. Canonicals, redirects, robots directives,
noindex, sitemap behaviour, URL structure, pagination, hreflang, authentication boundaries, and structured data describing real-world entities are all classifiedHIGHimpact, and none of them can beSAFE_AUTOMATIC.
SECURITY.md covers the prohibitions — cloaking, doorway pages, fabricated markup, keyword stuffing, paywall and authentication bypass.
npm run bench runs every case in benchmarks/cases/ and reports precision,
recall, false-positive rate, false-negative rate, and verification accuracy.
These are measurements of the checked subset of the registry against fixture projects containing defects we planted. They are a regression harness. They are not a measurement of real-world outcomes, and a perfect score on them says only that the checks behave as their cases declare.
The methodology, the definition of each denominator, and the reason a check that could not run is counted as neither a hit nor a miss are all in BENCHMARKS.md.
- SKILL.md — the agent orchestrator instructions
- AUDIT.md — the audit workflow, statuses, and safety model
- LIMITATIONS.md — what this repository cannot tell you
- BENCHMARKS.md — benchmark methodology and how to read a score
- EVIDENCE.md — generated: every citation, tier, and check date
- ARCHITECTURE.md — data model, pipeline, and module boundaries
- INTEGRATION.md — using the skill from an agent or from CI
- CONTRIBUTING.md — adding or changing a requirement
- requirements/registry.md — generated: domain index
- references/ — crawler policy, taxonomy, adapters, verification
- CHANGELOG.md — version history, including breaking changes
MIT.