Skip to content

Phase 4: --bakeoff CLI, backfill/docs polish (#9), AO3 vocab seed (#8)#19

Merged
elfensky merged 2 commits into
developfrom
desloppify/phase4-polish
Jul 10, 2026
Merged

Phase 4: --bakeoff CLI, backfill/docs polish (#9), AO3 vocab seed (#8)#19
elfensky merged 2 commits into
developfrom
desloppify/phase4-polish

Conversation

@elfensky

Copy link
Copy Markdown
Owner

Phase 4 of the desloppify code-health roadmap. Closes the last two open roadmap issues. Full suite green (42 core tests).

#9 (final items) — 68a6619

  • scourgify classify --bakeoff exposes the engine comparison that was locked in the wizard (display-only, writes nothing). Engine metadata (ENGINE_ENV + usable_engines()) now lives in classify as the single source of truth; wizard.ENGINE_KEYS aliases it so the two can't disagree.
  • Backfill preview shows each book's title, not just a raw id.
  • README engine list now includes mistral (already present in PRICING/ENGINE_ENV/classify).

#8 — seed the controlled vocab from AO3 (0fd851c)

The curated vocab was ~120 terms while wrangle ships the full ~93k-fold AO3 taxonomy, so the classifier under-tagged out of the box. The highest-use AO3 freeforms are the generic reusable tropes (Fluff, Angst, Slow Burn, Hurt/Comfort…) — a use-count threshold cleanly separates them from fandom-specific noise (which never accumulates many uses).

  • New maintainer tool build_classify_seed.py reads data/ao3_vocab.csv (name,uses, from the OTW dump via build_ao3_layer.py), gates at ≥20k uses, drops fandom-specific (...) and format/meta tags, and writes the regenerable defaults/classify_vocab_ao3.txt (~120 terms).
  • load_vocab() now merges curated core ∪ AO3 seed (dedup-on-merge, curated spelling wins), still fully overridable (-term trims a seeded term too).
  • The merge-dedup test surfaced and fixed a pre-existing Found Family duplicate in the curated file.
  • Verified the new seed ships in the wheel; data/ does not leak.

🤖 Generated with Claude Code

elfensky and others added 2 commits July 10, 2026 18:16
…fill + docs

Three verified polish items from the #9 batch:
- --bakeoff: the same sample-books-across-engines comparison the wizard offers is
  now available as `scourgify classify --bakeoff` (display-only, writes nothing).
  Engine metadata (ENGINE_ENV + usable_engines) now lives in classify as the single
  source of truth; wizard.ENGINE_KEYS aliases it so the two can never disagree.
- backfill preview now shows each book's title, not just its raw id.
- README engine list includes mistral (it was already in PRICING/ENGINE_ENV/classify).

Closes #9.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… freeforms

The curated vocab was ~120 terms while wrangle ships the full ~93k-fold AO3
taxonomy, so the classifier under-tagged out of the box (issue #8). The highest-use
AO3 freeforms ARE the generic reusable tropes (Fluff, Angst, Slow Burn, Hurt/Comfort…)
— a use-count threshold cleanly separates them from fandom-specific noise, which never
accumulates many uses.

build_classify_seed.py (maintainer tool) reads data/ao3_vocab.csv (name,uses; from the
OTW dump via build_ao3_layer.py), gates at >=20k uses, drops fandom-specific '(...)' and
format/meta tags, and writes the regenerable defaults/classify_vocab_ao3.txt (~120 terms).
load_vocab() now merges curated core ∪ AO3 seed (dedup-on-merge, curated spelling wins),
still fully overridable. Also removes a pre-existing 'Found Family' dup the merge-dedup
test surfaced.

Closes #8.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@elfensky
elfensky merged commit 51090a0 into develop Jul 10, 2026
2 checks passed
@elfensky
elfensky deleted the desloppify/phase4-polish branch July 10, 2026 16:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant