Skip to content

Upstream the concept ledger, add prose and duplication gates, fix three silent-staleness bugs - #26

Merged
alpibrupa merged 8 commits into
mainfrom
upstream-the-concept-ledger
Sep 3, 2026
Merged

Upstream the concept ledger, add prose and duplication gates, fix three silent-staleness bugs#26
alpibrupa merged 8 commits into
mainfrom
upstream-the-concept-ledger

Conversation

@alpibrupa

Copy link
Copy Markdown
Contributor

Closes #9, #10, #11.

The ledger comes upstream

All three of those issues were prototyped as a scripts/ directory inside prompt-to-production and then duplicated byte for byte into three sibling books — which is how the case for moving them here got demonstrated rather than argued. 292 lines × 4 copies, and the most recent fix to them had to be applied four times.

content_kit_core.ledger gains the Concept/LedgerConfig model, next to the story bible, because it is canon and core owns canon. bookkit gains the rules that read a manuscript, because it is the tier that knows what a chapter is. The generated glossary reproduces the old script's output byte for byte — verified by diffing against the version reconstructed from git.

The matching is stricter than what it replaces. A gate that matches "dependency" but not "dependencies" is not a stricter gate, it is a gate with invisible holes. Fixing that immediately surfaced two real errors the old scripts could not see.

What else is here

check prose the house-style linter — a reader found a comma before every restrictive because-clause, 81 across the four books
check duplication how much of one book is already in a sibling, compared every-chapter-to-every-chapter so a moved chapter is still found
audiobook diagrams were being narrated as XML — 123 lines, ~59 minutes across the series
podcastkit voice lines now track the text they were rendered from; assembly stops leaving 2.0 GB of scratch behind
front_matter accepts a file, so a preface or series map can open a book
pytest the workspace runs in one pass; the README's own command could not work before

Notes for review

Three of these are bugs found by using the toolchain rather than reading it, and each was mis-diagnosed once before being fixed properly — the details are in the individual commit messages, which are worth reading over the diff:

  • The voice-line cache fix originally left pre-existing audio permanently unverifiable, so the bug it was meant to fix survived for every existing project.
  • write_project was silently regenerating nothing after a manuscript edit; the first measurement of drift said "0 lines changed" after five chapters had been rewritten, which is what gave it away.
  • A test caught me making episode.yaml content-aware alongside script.json — it is the hand-tuned cast sheet and must survive regeneration. That distinction is now explicit in the code.

281 tests (up from 156), 4/4 import contracts kept, ruff clean.

Merge order

The four book repositories have PRs that depend on this one: their Makefiles and CI call bookkit check terms, check prose and glossary. This must merge first, or every book's CI fails on a missing command.

🤖 Generated with Claude Code

alpibrupa and others added 8 commits September 3, 2026 15:11
…ated glossary

Closes content-kit#9, #10 and #11. All three were prototyped as a scripts/
directory inside prompt-to-production and then duplicated, byte for byte, into
three sibling books -- which is how the case for moving them here got
demonstrated rather than argued.

core gains the Concept/LedgerConfig model, next to the story bible, because it
is canon and core owns canon. bookkit gains the rules that read a manuscript,
because it is the tier that knows what a chapter is.

The matching rules are stricter than the scripts they replace. A gate that
matches "dependency" but not "dependencies" is not a stricter gate, it is a
gate with invisible holes: name_pattern now handles regular and common
irregular plurals, the three spellings of a hyphenated term
(trade-off/trade off/tradeoff), and line breaks inside a multi-word term.
Signposts are recognised in the forms authors actually write -- "Chapters 6 and
7", "Chapters 7 through 10" -- because a gate that rejects a properly
signposted forward reference punishes the writing it exists to encourage.

New rules beyond the originals: dependency-cycle, self-dependency,
concept-missing-definition, defined-in-missing (a term with no chapter silently
disabled its own check), defined-in-out-of-range, scan-name-unknown,
chapter-defines-nothing, and chapter-reference-out-of-range -- the last of
which catches the renumbering damage that inserting a chapter leaves behind,
and keeps the signpost escape hatch honest.

scan: now also takes a list, so a concept whose alias is ordinary English
("sanity check") can scan its distinctive name alone rather than giving up the
check entirely.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A reader spotted a comma before every restrictive because-clause, which is a
Spanish punctuation habit carried into English -- 81 instances across the four
books. It is mechanical enough to check, so it should not be something a reader
has to notice.

Findings are warnings and never errors: a style gate that can block a build
will eventually block one for a sentence that was right, and then it gets
disabled.

Two rules. comma-before-because is deliberately conservative -- it stays quiet
when the main clause is negative (where the comma changes the meaning), when an
em dash has already opened an aside, and on elliptical afterthoughts. The
second started as "because twice in one sentence", which found 16 hits of which
15 were legitimate correlatives ("not because X, but because Y") and parallel
list items; narrowed to the one shape that actually trips a reader.

--fix applies the corrections that need no judgement.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A series that carries one method across volumes will reuse some prose with it,
and some of that is right. What is not right is a chapter a reader recognises
-- same shape, same sentences, the domain nouns swapped -- because the second
reading then feels like a find-and-replace of the first.

Sentence-level overlap, and every chapter is compared against every chapter of
the sibling rather than the one with the same number, which is what catches a
shared chapter that has moved between volumes. On the two books in this series
that share a method it finds one such pair the same-numbered comparison misses
entirely.

The comparison is deliberately literal -- no synonym table, no stemming -- so
the number falls on its own the moment a sentence is genuinely rewritten. A
test asserts exactly that, so the measure cannot quietly stop measuring the
thing anyone cares about.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…uscript

Three bugs, all found by listening to what the pipeline actually produced.

Inline SVG diagrams were narrated as XML. markdown_to_speech stripped code
fences, links, headings and lists but had no rule for raw HTML, so 123
narration lines across the four books were "div style margin one point six rem,
svg viewBox zero zero six eighty one twenty, xmlns http colon slash slash..."
read aloud. Chapter 1 of Production alone carried 4m44s of it.

Markdown tables were narrated with their pipes and separator rules intact.
Unlike a diagram, a table's cells carry the content rather than the packaging,
so each row is now narrated as its cells in order rather than dropped.

Together those removed 56,749 characters -- about 59 minutes of markup that
would have been read aloud across the series.

And write_project only wrote script.json when the file did not already exist,
so `bookkit audiobook` after an edit regenerated nothing, said nothing, and the
stale script was then narrated as though current. Now content-aware.
episode.yaml deliberately is not: it is the cast sheet an author hand-tunes
with real voices, and clobbering that on every regeneration would be its own
bug.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… assembly

A voice line is cached as voices/<line_id>.mp3 and a line id survives an edit
to its text, so correcting a sentence left the old audio in place -- silently,
permanently, and looking like a successful render. A manifest now records a
hash of the words each file was rendered from: edit a sentence and only that
line re-renders.

A first version skipped lines with no manifest entry, reasoning that
re-rendering everything could be a surprise bill on a paid backend. Testing it
against a real chapter showed the edited line was still skipped -- unverified
lines stayed unverified forever, so the original bug survived for every
existing project. Pre-existing audio is now adopted on first sight and recorded
against the script as it currently reads, which costs nothing and makes the
next edit detectable.

assemble left its uncompressed pass-1 WAV behind: 2.0 GB across a 17-chapter
book, for a file nothing reads once the MP3 verifies. Now removed on success,
kept on failure, and kept by --keep-intermediate when debugging a mix.

The kokoro backend also grew a check for espeak-ng's 160-byte data-path limit,
which silently falls back to a build-time path and reports a missing file on a
machine where nothing is missing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Only back matter could carry a file entry, which stranded material a reader
needs before chapter 1 at the end of the book. All four books in the Prompt
to... series shipped their one-page series map -- the page that explains what
"the second book in this series" refers to -- in back matter, roughly 150 pages
after the first cross-reference that needs it.

Nothing makes a preface less legitimate than an afterword, so the asymmetry
looks accidental. Front matter now takes the same MatterEntry back matter does,
in both the HTML and EPUB paths.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every package carried an empty tests/__init__.py, so all four imported as the
same top-level module names (tests.conftest, tests.test_cli) and collided the
moment more than one package was collected in a single run. The README's own
`pytest packages/core packages/bookkit packages/podcastkit` could not work.

Removed those files and set importlib import mode at the workspace root, next
to the shared ruff and import-linter config. `pytest` from the root now runs
every package; each still runs standalone.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
check terms, check prose, check duplication and glossary, each registered in
the introspect catalog so the MCP server exposes them without drifting, and
documented in bookkit's README and the workspace README.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Genre-aware canon: concept and notation ledgers alongside the story bible

1 participant