Phase 4: --bakeoff CLI, backfill/docs polish (#9), AO3 vocab seed (#8)#19
Merged
Conversation
…fill + docs Three verified polish items from the #9 batch: - --bakeoff: the same sample-books-across-engines comparison the wizard offers is now available as `scourgify classify --bakeoff` (display-only, writes nothing). Engine metadata (ENGINE_ENV + usable_engines) now lives in classify as the single source of truth; wizard.ENGINE_KEYS aliases it so the two can never disagree. - backfill preview now shows each book's title, not just its raw id. - README engine list includes mistral (it was already in PRICING/ENGINE_ENV/classify). Closes #9. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… freeforms The curated vocab was ~120 terms while wrangle ships the full ~93k-fold AO3 taxonomy, so the classifier under-tagged out of the box (issue #8). The highest-use AO3 freeforms ARE the generic reusable tropes (Fluff, Angst, Slow Burn, Hurt/Comfort…) — a use-count threshold cleanly separates them from fandom-specific noise, which never accumulates many uses. build_classify_seed.py (maintainer tool) reads data/ao3_vocab.csv (name,uses; from the OTW dump via build_ao3_layer.py), gates at >=20k uses, drops fandom-specific '(...)' and format/meta tags, and writes the regenerable defaults/classify_vocab_ao3.txt (~120 terms). load_vocab() now merges curated core ∪ AO3 seed (dedup-on-merge, curated spelling wins), still fully overridable. Also removes a pre-existing 'Found Family' dup the merge-dedup test surfaced. Closes #8. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 4 of the desloppify code-health roadmap. Closes the last two open roadmap issues. Full suite green (42 core tests).
#9 (final items) —
68a6619scourgify classify --bakeoffexposes the engine comparison that was locked in the wizard (display-only, writes nothing). Engine metadata (ENGINE_ENV+usable_engines()) now lives inclassifyas the single source of truth;wizard.ENGINE_KEYSaliases it so the two can't disagree.mistral(already present inPRICING/ENGINE_ENV/classify).#8 — seed the controlled vocab from AO3 (
0fd851c)The curated vocab was ~120 terms while
wrangleships the full ~93k-fold AO3 taxonomy, so the classifier under-tagged out of the box. The highest-use AO3 freeforms are the generic reusable tropes (Fluff, Angst, Slow Burn, Hurt/Comfort…) — a use-count threshold cleanly separates them from fandom-specific noise (which never accumulates many uses).build_classify_seed.pyreadsdata/ao3_vocab.csv(name,uses, from the OTW dump viabuild_ao3_layer.py), gates at ≥20k uses, drops fandom-specific(...)and format/meta tags, and writes the regenerabledefaults/classify_vocab_ao3.txt(~120 terms).load_vocab()now merges curated core ∪ AO3 seed (dedup-on-merge, curated spelling wins), still fully overridable (-termtrims a seeded term too).Found Familyduplicate in the curated file.data/does not leak.🤖 Generated with Claude Code