Distribution profiles: resolve overlays against a harness, re-measure both - #55
Merged
Conversation
Revisions 1 through 8 required identical membership everywhere, on the reasoning that per-harness membership would change profile identity semantics. The concern was right and the conclusion was not: it is resolved by separating two identities that had been conflated. Profile identity is intent and stays harness-independent; realization identity is what landed and already bound the harness. Resolved membership moves from the first to the second. The mechanism is narrow on purpose. Not free-form per-harness membership but one declared rule with one trigger -- the harness ships its own equivalent of a member -- declared per skill with evidence, never inferred from a name match. An undeclared collision is reported rather than suppressed, because the failure mode of guessing is losing a skill the maintainer wanted. The evidence is live rather than anticipated. Codex bundles six skills as .system entries and Claude Code bundles a different three with no overlap, so dojo's skill-creator is duplicated in every Codex session today. The harnesses also differ in budget by 3-5x, in scope precedence, and in degradation mode. Identical membership had stopped being a simplification and become a misstatement. SC-02 gains the distinction that anchors constrain the definition rather than the realization, SC-11 scopes comparability to a shared harness and requires every cross-harness difference to name what displaced the member, and EV-NEG-06 takes the scenario count to 16.
Follows spec revision 9. The structural change is in identity: profile_identity drops resolved membership and hashes intent alone, so a member suppressed on one harness no longer makes two harnesses look like different profiles. Membership moves to realization_identity, which gains equivalence_identity as a sixth field. Task 1 gains profiles/harness-equivalences.yaml -- one entry per (skill, harness) pair with the bundled entry and the evidence for the claim, seeded from live observation. Task 2 gains resolve_for_harness returning realized and suppressed sets, with an undeclared name collision returned as a collision rather than a suppression, because guessing silently removes a skill the maintainer selected. Task 5's evidence carries the suppressed list so a suppressed member is distinguishable from one the profile never selected. That distinction is the point of declaring suppression rather than editing membership. Task 7 is where this could have gone wrong. Membership agreement now applies only when both sides name the same harness; across harnesses every difference must be attributable to a suppression naming what displaced it, and an unattributed difference is drift. Without that check "different harness" becomes a blanket excuse hiding the divergence profiles exist to catch. EV-NEG-06 traces to Tasks 1, 2, 5, and 7. All 13 SC and 16 EV ids trace.
…docs Spec revision 10. The #53/#54 merges moved the Codex figures by roughly a factor of two, and the spec and plan were still arguing from the pre-merge numbers. Codex now truncates nowhere: ordinary session 96% -> 78%, dojo 177% -> 78% (95 entries -> 41), viral 111% with 19 truncated -> 97% with none. Two of the three Codex breaches turned out to be defects rather than distribution problems, and were fixable without profiles. That is recorded plainly rather than glossed: what survives is that nothing proposed, measured, or now prevents the regression of any of it, and viral remains non-deployable at 97% against the 90% ceiling. Claude Code is where the argument now rests. The same levers moved it from 2.13x to 2.07x, because none of them touched membership -- description trimming recovered 537 chars against an 8,535-char gap. A 45-entry listing against 8,000 chars allows ~178 chars per description while this repo's own skill contract requires descriptions to carry trigger conditions. Two live subjects re-checked rather than assumed. Cross-machine agreement had silently broken across 9 skills since revision 9 -- membership unchanged, content identity diverged, nothing reporting it -- and was restored by an explicit sync on the mini; both docs now treat it as a subject needing re-establishment, not a standing fact. microsoft-foundry is installed but disabled, so it is no longer a live foreign-entry observation and that fixture must be constructed. Also: catalog count 49 -> 48 (fourth stale count in this pair, now stated with its date and computed source), and Task 13 flagged as drafted against pre-#54 code -- HARNESS_DIRS no longer contains .agents and every line number in it shifted. No success criterion, evaluation scenario, or authority boundary changed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Spec revisions 9 and 10, plus the matching plan revisions. Contract-only — no code changes.
Revision 9 — overlays resolve against a harness
Revisions 1–8 forbade per-harness membership because it "would change profile identity semantics". The concern was right and the conclusion was not. Fixed by separating two identities that had been conflated:
The mechanism is deliberately narrow: one declared rule with one trigger — the harness ships its own equivalent — declared per canonical skill with evidence, never inferred from a name match. An undeclared collision is reported, never silently suppressed, because the failure mode of guessing is losing a skill the maintainer selected.
Live justification:
skill-creatoris duplicated in every Codex session today.EV-NEG-06added, scenarios 15 → 16.Revision 10 — re-measured after #53/#54
The merges moved the Codex numbers by roughly 2× and both docs were still arguing from pre-merge figures.
dojoviraldojoTwo of the three Codex breaches were defects, not distribution problems, and were fixable without profiles. That is stated plainly rather than glossed. What survives: nothing proposed, measured, or now prevents the regression of any of it, and
viralis still non-deployable at 97% against the 90% ceiling.Claude Code is now the binding harness. The same levers moved it a few percent because none touched membership — description trimming recovered 537 chars against an 8,535-char gap. A 45-entry listing against 8,000 chars allows ~178 chars per description, while this repo's skill contract requires descriptions to carry trigger conditions. Nothing is left to trim.
Two live subjects re-checked rather than assumed
microsoft-foundryis installed but disabled in Codex config. Present on disk, absent from the listing — a good live case for "the filesystem never adds an entry the probe did not list", but not a live foreign-entry observation. Task 4 must construct that fixture.Corrections
algorithmic-artretired). Fourth stale count in this document pair; now stated with its date and computed source rather than as a bare number. The plan's test was already correct — it asserts againstlen(skills.json), not a literal — only the prose had drifted.HARNESS_DIRSno longer contains.agents,retire_legacy_symlinknow exists, and every line number in that task shifted.No success criterion, evaluation scenario, or authority boundary changed in either revision.
Verification
validate_spec.pyPASS,validate_plan.pyPASSpytest tests/ -q→ 313 passedcodex debug prompt-inputandclaude --debug-file, not carried forward🤖 Generated with Claude Code