spec: seed the 4.3 cycle with character content for the corpus form - #89
Conversation
…icholas-Keystate#86) Section 15 already makes "the bytes fail the committed corpus form" a conviction kind and glosses the form as ordering and corpus identity. Nothing in the ratified text says which characters may appear, so a clause can read one way to every reviewer and hash as something else. The seed gives the form character content in a portable shape: a domain commits its script set, and a character outside it is a canonical-form violation. Unconditionally excluded are the two bidirectional overrides, three invisibles, unpaired surrogates, control characters other than tab and newline, and text not in NFC. U+200B stays conditional, because Thai, Lao, Khmer and Myanmar use it to divide words. Refusal rather than repair, since a stripped override yields a clause nobody enacted. tools/check-corpus-characters.py enforces the unconditional part plus an allowlist of the 10 non-ASCII characters this corpus actually uses, none of which is a letter — which defeats every cross-script homoglyph without naming a script or consulting Unicode data. The whole corpus passes today. Confusable detection is left out on purpose. Per-word script mixing fires on Japanese and misses an all-Cyrillic word that renders as Latin; the relational test needs Unicode data the standard library lacks. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Daniel Hardman <daniel.hardman@gmail.com>
Nicholas-Keystate
left a comment
There was a problem hiding this comment.
Substance-gate review at the branch bytes.
Checker verified: check-corpus-characters.py runs green here as claimed — 31 files, 0 failures; the corpus arrives conforming, so this is prevention at its cheapest moment. The claim that the ratified bytes contain no charset/normalization language was re-verified by search.
The shape is right by this record's own laws. The committed script set is the declare-don't-mandate pattern (custody plurality, S7-2, S8-1, and the S9 coincidence ruling all take the same form): the standard mandates that a domain commits, never what. The unconditional exclusions are correctly minimal (renderer-instructions only; bidi marks and ZWSP conditionally spared for the scripts that need them — the Azerbaijani/Japanese/Thai reasoning is the right test). Refusal-not-repair follows §15's existing conviction shape: bytes silently corrected are a clause nobody enacted — that sentence is spec-grade and should survive into the successor's text.
Enforcement-grade note for the S8-1 register: this covenant is act-detectable — violation derives from committed bytes alone, no promoting instrument needed. The declaration writes itself.
One pin owed: the normative text references Normalization Form C and named codepoints; when spec text lands at 4.3, UAX #15 / the Unicode version should enter the external-pin discipline (engagement-companion route) like every other consumed external semantic — otherwise "NFC" floats on whatever Unicode revision is ambient, which is the defect class the seed itself is curing one level down.
Ruling posture: merging as seed-intake to the drafting corpus; the ruling the header confesses as owed executes with the 4.3 chain per the docket's standing pattern. Script-set placement (§18 vs founding law) stays chartered to the re-rooting decision, as the seed correctly leaves it.
A 4.3 seed giving §15's committed corpus form some content about characters, plus the tool that enforces it.
Section 15 already makes "a canonical-form violation (the bytes fail the committed corpus form — ordering, corpus identity)" a conviction kind (L2777-2778). So the form exists and failing it is convictable. Nothing says which characters may appear — searched 2026-08-26, the ratified bytes contain no occurrence of "unicode", "utf-8", "codepoint", "charset", or any form of "normaliz". Finding #86 carries the evidence and the attacks.
The seed proposes that a domain commit its script set, with a character outside it a canonical-form violation. That is what makes the rule portable: Custos cannot say which scripts are legitimate without being wrong about Azerbaijani or Japanese, and a committed set is derivable from bytes with no Unicode data file. Independently of the set, law bytes are in NFC and exclude the two bidirectional overrides, three invisibles, unpaired surrogates, and control characters other than tab and newline.
U+200Bstays conditional, because Thai, Lao, Khmer and Myanmar use it to divide words. Refusal rather than repair, since a stripped override yields a clause nobody enacted.tools/check-corpus-characters.pyenforces the unconditional part over every file inspec/andlineage/plus the three root documents, together with an allowlist of the 10 non-ASCII characters this corpus actually uses — none of which is a letter, which is what defeats every cross-script homoglyph without naming a script. The corpus passes today: 748,199 characters, 30 files, nothing to fix. This is prevention rather than cleanup.Confusable detection is deliberately absent. Per-word script mixing fires constantly on Japanese and misses an all-Cyrillic word that renders as Latin; the relational test needs Unicode data the standard library does not carry, and that dependency is a decision the seed does not make.
This does not discharge #86. A ruling is owed before the seed can enter. Offered to the #77 docket.
Relates to #86, #77, and #85.