Extract deep, structured lorebooks from novels — deterministic pipeline, AI at checkpoints, anti-hallucination validation.
MennzLore converts any novel (Project Gutenberg, EPUB, Standard Ebooks) into a comprehensive lorebook covering characters, relationships, timeline, world state, cinematography shot lists, SVG maps, and more. The engine is fully deterministic; an LLM (Claude via MCP) participates only at two well-defined analysis gates.
| Output | Location |
|---|---|
| Master lorebook (Markdown) | output/<prefix>_master_lorebook_full.md |
| Per-chapter micro-facts (JSON) | micro_facts/<prefix>_EP###_micro_facts.json |
| Cinematography shot list | output/production/cinematography_shot_list.json |
| Image prompts (Midjourney / SD) | output/production/scene_image_prompts.json |
| Visual style bible | output/production/visual_style_bible.json |
| Entity registry | output/production/entity_registry.json |
| SVG world map | output/spatial/chart_map_skeleton.svg |
| Relationship graph | output/entities/relationship_graph.svg |
| Timeline SVG | output/entities/timeline.svg |
| SQLite knowledge graph | output/knowledge.db |
- Python 3.11 or 3.12
- Claude Desktop (or any MCP-capable AI client) for the LLM analysis phases
- Git
git clone https://github.com/mgprona/MennzLore.git
cd MennzLore
pip install -e .For the MCP server (Claude Desktop integration):
python install.py# Download from Project Gutenberg by title + author
python engine/fetch_raw.py "The Sign of Four" "Arthur Conan Doyle" ./projects
# Split into chapters
python engine/split_chapters.py ./projects/sign-of-four-doyleOutput:
raw/<prefix>_full.txt— full raw textclean/<prefix>_EP###.txt— one file per chapterverification/<prefix>_chapters.json— chapter manifest
Open Claude Desktop and run these MCP tools in order:
1. run_global_lore_extraction → character list, name map, timeline framework
2. analyze_chapter (EP001) → micro-facts for chapter 1
analyze_chapter (EP002) → repeat for every chapter
3. merge_micro_facts → merge + hallucination validation
python engine/assemble_generic.py ./projects/sign-of-four-doyle
python engine/assemble_production_generic.py ./projects/sign-of-four-doyle
python engine/chart_render_generic.py ./projects/sign-of-four-doyle
python engine/relationship_graph.py ./projects/sign-of-four-doyle
python engine/timeline_render.py ./projects/sign-of-four-doyle
python engine/entity_registry.py ./projects/sign-of-four-doyleYour master lorebook:
projects/sign-of-four-doyle/output/<prefix>_master_lorebook_full.md
| Phase | Script | LLM? | What it does |
|---|---|---|---|
| 1. Acquire | engine/fetch_raw.py |
✗ | Download from Gutenberg / EPUB / Standard Ebooks |
| 2. Split | engine/split_chapters.py |
✗ | Detect headings, write per-chapter files |
| 3. Global lore | engine/phase3_global_lore.py |
✓ | Character list, name map, timeline framework |
| 4. Micro-facts | MCP analyze_chapter |
✓ | 2- or 3-pass analysis per chapter |
| 5. Merge | engine/merge_to_micro_facts.py |
✗ | Merge passes, validate cross-refs |
| 6. Production | engine/assemble_production_generic.py |
✗ | Shot lists, image prompts, style bible |
| 7. Spatial | engine/chart_render_generic.py |
✗ | SVG world map |
| 8. Relationships | engine/relationship_graph.py |
✗ | Force-directed graph SVG |
| 9. Timeline | engine/timeline_render.py |
✗ | SVG timeline + heatmap |
| 10. Entities | engine/entity_registry.py |
✗ | Typed entity registry |
| 11. Hybrid notes | engine/hybrid_notes.py |
✗ | Per-entity combined notes |
| 12. Knowledge | engine/knowledge_graph.py |
✗ | SQLite FTS5 knowledge graph |
| 13. Semantic | engine/vector_rag.py |
✗ | TF-IDF search index |
| 14. Assemble | engine/assemble_generic.py |
✗ | Master lorebook Markdown |
Adaptive token cost: chapters under 15,000 chars use a 2-pass LLM run (Architect+Profiler combined, then Chronicler). Longer chapters get 3 passes. This cuts token usage by 60–70% on typical novels.
Anti-hallucination: every LLM output carries a _source_hash validated against the source text. Pydantic cross-field validators check every in_scene_id reference points to a real extracted scene. Mismatches raise HallucinationError and block the merge step.
| Tool | Description |
|---|---|
acquire_by_id |
Download by Gutenberg ID |
acquire_by_title |
Search and download by title + author |
acquire_epub |
Import a local EPUB |
split_chapters |
Phase 2 — split into chapters |
run_global_lore_extraction |
Phase 3 — global lore |
analyze_chapter |
Phase 4 — micro-facts for one chapter |
merge_micro_facts |
Phase 5 — merge + validate |
run_production_render |
Phase 6 — cinematography |
run_map_render |
Phase 7 — SVG map |
run_relationship_graph |
Phase 8 — relationship graph |
run_timeline_render |
Phase 9 — timeline |
query_knowledge_graph |
FTS5 search |
run_assemble_lorebook |
Phase 14 — master lorebook |
pipeline_status |
Show phase progress |
| Source | Command |
|---|---|
| Project Gutenberg | python engine/fetch_raw.py "<title>" "<author>" <dir> |
| Standard Ebooks | python engine/acquire_se.py "<slug>" <dir> |
| EPUB file | python engine/fetch_epub.py <path/to/book.epub> <dir> |
# Install with dev dependencies
pip install -e ".[dev]"
# Run unit tests
python -m pytest tests/unit/ -v
# Lint
ruff check mennzlore/ interfaces/CI: Python 3.11 + 3.12, 215/215 tests passing.
MennzLore/
├── engine/ # Fully functional legacy scripts (phases 1–14)
├── mcp_server/ # MCP server — use this with Claude Desktop today
├── mennzlore/ # v3 Python package (refactor in progress)
│ ├── core/ # errors, paths, models, classifiers, io
│ ├── pipeline/ # acquire, split, merge, render, assemble, runner
│ ├── saga/ # multi-volume support
│ └── prompts/ # PromptChain + template loader
├── interfaces/
│ ├── mcp/ # New modular MCP (in progress)
│ └── cli/ # CLI entry point (basic)
├── plugins/ # Optional: youtube, image_gen, translation, dashboard
├── tests/
│ ├── unit/ # Unit tests for mennzlore.* (215 tests)
│ └── (root) # Legacy tests for engine.* (81 tests)
├── prompts/ # Prompt templates (.md + YAML frontmatter)
├── schemas/ # JSON schemas for structured LLM output
└── docs/ # ENGINE.md, PIPELINE.md, MCP_DESIGN.md
| Component | Status |
|---|---|
engine/ scripts (phases 1–14) |
✅ Fully functional |
mcp_server/server.py |
✅ Fully functional |
mennzlore.core.* |
✅ Complete, tested |
mennzlore.pipeline.* (phases 5–14) |
✅ Implemented |
Pipeline.run() phases 1–4 |
🔧 Stubs — use engine/ scripts for now |
interfaces/cli/ |
🔧 Basic only |
| Plugins | 🔧 Extracted, partial |
- Wire phases 1–4 into
Pipeline.run() - Connect
CacheManagerto_run_micro_facts - Full
interfaces/cli/with click subcommands - Integration tests against a small test novel
- Publish to PyPI
MIT — see LICENSE.