Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion conformance/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,7 @@ identical corpus.
|---|---|---|
| `parser` | The Musher YAML profile — [component §7.1](../specifications/component/v1/spec.md#yaml-profile) | Never |
| `structural` | The family's JSON Schema 2020-12 bundle | Never |
| `semantic` | Reference resolution, path containment, dependency cycles | Never |
| `semantic` | Reference resolution, path containment, cross-document agreement | Never |
| `capability` | Account, region, and quota checks | Server only |

An implementation MUST apply the phases in order and MUST NOT report a
Expand Down
13 changes: 8 additions & 5 deletions specifications/blueprint/v1/spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -748,11 +748,14 @@ reviewer of the item can see: a graph that reaches outside the directory being
reviewed deploys something the review did not cover.

**Graph traversal.** [§4.2](#connections) makes the component graph a directed
graph an implementation walks. A cycle is rejected with `ERR_CONNECTION_CYCLE`,
and an implementation MUST detect cycles rather than relying on a recursion limit
to stop it — a stack overflow is a crash, not a diagnostic. The parser's nesting
bound does not help here: the cycle is in the graph the document describes, not
in the document's own structure.
graph an implementation walks, and permits that graph to contain a cycle. An
implementation MUST therefore detect cycles rather than relying on a recursion
limit to stop it — a stack overflow is a crash, not a diagnostic. Detecting one
means terminating the walk, not rejecting the document: that clause forbids
rejecting a composition for containing a cycle, so a traversal that meets one
MUST finish rather than report. The parser's nesting bound does not help here:
the cycle is in the graph the document describes, not in the document's own
structure.

**Published references.** Resolving a published reference is `capability`
([§6](#validation-layers)) precisely because it needs the catalog. An
Expand Down
72 changes: 72 additions & 0 deletions tools/src/conformance.ts
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,14 @@ const BASE_FAMILY = 'component'
const DIAGNOSTIC_ROW = /^\|\s*`(ERR_[A-Z0-9_]+)`\s*\|\s*`([a-z]+)`\s*\|/
/** A stable heading anchor, `## <a id="envelope"></a>2. Document envelope`. */
const SPEC_ANCHOR = /<a id="([^"]+)"><\/a>/g
/**
* A diagnostic code named anywhere in the prose, inside backticks.
*
* `DIAGNOSTIC_ROW` is anchored to `^|`, so it sees only the registry tables. A
* code named in a sentence is the same promise to an implementer and was
* matched by nothing — see `checkProseCodes`.
*/
const PROSE_CODE = /`(ERR_[A-Z0-9_]+)`/g

interface SpecIndex {
/** Diagnostic code to the phase the prose assigns it. */
Expand Down Expand Up @@ -197,6 +205,23 @@ function registryFor(family: Family): ReadonlyMap<string, Phase> {
return new Map([...(base ?? EMPTY_INDEX).codes, ...own.codes])
}

/**
* Every code any of the three registries declares.
*
* Deliberately global rather than `registryFor`'s reachable set. A family's
* prose legitimately names another family's code — component §3 and §10 name
* blueprint's `ERR_UNKNOWN_COMPONENT`, listing §4 names
* `ERR_UNREFERENCED_COMPONENT` — and those are citations, not declarations.
* What `checkProseCodes` asks is whether the code exists at all.
*/
function declaredCodes(): ReadonlySet<string> {
const codes = new Set<string>()
for (const family of discoverFamilies()) {
for (const code of specIndex(family.specPath)?.codes.keys() ?? []) codes.add(code)
}
return codes
}

function loadIndex(family: Family, failures: Failures): CaseIndexEntry[] {
const indexPath = join(family.conformanceDir, 'cases.json')
if (!existsSync(indexPath)) return []
Expand Down Expand Up @@ -605,6 +630,52 @@ function checkRequirementCoverage(cited: ReadonlySet<string>, failures: Failures
}
}

/**
* Codes the prose names on purpose without declaring them, and why.
*
* The same shape as `UNCOVERED` and `UNPINNED`, for the same reason. An entry
* here is a claim a reviewer can check, and the list should stay near empty:
* naming a code that does not exist is how a withdrawn rule survives.
*/
const HYPOTHETICAL: ReadonlyMap<string, string> = new Map([
[
'ERR_SCHEMA_TOO_OLD',
'component §3 names it as a code that deliberately does not exist, to explain why a field from a newer release is reported as ERR_UNKNOWN_FIELD — a validator holding neither definition cannot tell that case from a misspelling',
],
])

/**
* Every diagnostic code the prose names is declared by a registry.
*
* The third direction, and the one nothing checked. `checkCaseShape` asks
* whether a code a *fixture* declares exists; `checkCoverage` asks whether a
* code a *registry* declares is fixtured. Neither looks at a code named in a
* sentence — which is how blueprint §10 came to reject a cycle §4.2 permits,
* with `ERR_CONNECTION_CYCLE`, a code no table has ever defined, and CI green.
*
* Scoped to the three spec.md files. An ADR is immutable and records withdrawn
* codes as history, so a code that no longer exists is correct there; a named
* code is a promise only in a normative document.
*/
function checkProseCodes(failures: Failures): void {
const declared = declaredCodes()
for (const family of discoverFamilies()) {
if (!existsSync(family.specPath)) continue
const lines = readFileSync(family.specPath, 'utf8').split('\n')
for (const [offset, line] of lines.entries()) {
for (const match of line.matchAll(PROSE_CODE)) {
const code = match[1]
if (code === undefined || declared.has(code) || HYPOTHETICAL.has(code)) continue
failures.add(
`${relativeToRepo(family.specPath)}:${offset + 1}: ${code} is named in the prose but ` +
'is declared by no diagnostics table. Add it to a registry, or record it in ' +
'HYPOTHETICAL with a reason.',
)
}
}
}
}

/**
* Case directories that no `cases.json` entry names. The index is the contract
* and the runner never walks the filesystem, so an unindexed directory is not a
Expand Down Expand Up @@ -654,6 +725,7 @@ function main(): void {
}

checkRequirementCoverage(cited, failures)
checkProseCodes(failures)

const suffix = skipped > 0 ? ` (${skipped} skipped)` : ''
const profile = profileFor(IMPLEMENTED_PHASES) ?? 'none'
Expand Down