SunneV · SunneV · Jun 30, 2026 · Jun 30, 2026
diff --git a/.gitignore b/.gitignore
@@ -47,6 +47,7 @@ htmlcov/
 scriber_pack.md
 *.scriber_pack.md
 context.md
+audit*.html
 
 # IDEs and Editors
 .idea/

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -5,6 +5,50 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 
+## [2.2.0] - 2026-06-30
+
+A major upgrade driven by three rounds of code audit. Adds an interactive dependency-graph visualization, real graph algorithms, a parallel native scanner, three opt-in build-time features (BPE tokenizer, tree-sitter AST, graph snapshot), and numerous correctness/performance fixes. All new Cargo features are **off by default** and degrade gracefully when absent.
+
+### Added
+- **Interactive dependency-graph visualization**: `render_graph_html()` produces a self-contained, dependency-free HTML file with a Canvas force-directed layout. Auto-emitted to `.scriber/graph.html` next to every pack (`emit_graph_html = true`, suppress with `--no-graph-html`). Features: per-component gravity (uncorrelated languages form separate islands), radial/hierarchical init from dependency depth, hub emphasis, weight/confidence-aware springs, minimap, search, inertial pan, keyboard nav, hover tooltips, in-pack highlighting, and embedded Scriber branding.
+- **Graph export formats**: `--graph-dot` (Graphviz), `--graph-mermaid` (GitHub/Notion native), `--graph-html`, alongside the existing `--graph-json`. All render every relation kind, not just imports.
+- **Graph algorithms** (`engine/graph_algorithms.py`): Tarjan SCC for import-cycle detection, Kahn topological layering, weighted-degree centrality, and PageRank. Wired into `--explain-graph` (cycles, top hubs, top influential, architectural layers) and the ranker (centrality bonus).
+- **Symbol-level relations** (`type_reference`, `inherits`): emitted by the Python AST symbol extractor on the Python builder path, and by tree-sitter on the native path (when the `treesitter` feature is enabled).
+- **Java import extraction**: `parse_java_imports` / `resolve_java_import` resolve `com.example.Foo` to `src/main/java`/`src/test/java`.
+- **BPE tokenizer** (Cargo feature `bpe`): exact offline token counting via `tiktoken-rs` (cl100k_base / o200k_base). `TokenConfig.encoding` selects the encoding; `has_bpe_tokenizer()` probes availability; falls back to the calibrated estimator when absent.
+- **tree-sitter AST backend** (Cargo feature `treesitter`): walks the Python AST to emit `type_reference`/`inherits` edges on the native path where they were previously dead. Pluggable per-grammar.
+- **Graph snapshot + restore** (`graph/snapshot.py`): persists the whole RelationGraph to `.scriber/cache/graph.json`, restored on the next run when no file changed. Closes the native "write-only" cache defect.
+- **Configurable cache LRU** (`cache.max_entries`, default 50 000, `0` = unlimited): replaces the hard 1000-entry cap.
+- **Real budget enforcement**: every content mode consumes the budget with calibrated weights (`MODE_TOKEN_WEIGHT`); allocator respects a true `hard_limit`.
+- **LLM profiles in CLI**: `gpt`, `focused-gpt`, `full` exposed in `--profile` choices.
+- **Calibrated token estimation**: `estimator="auto"` selects a per-language chars-per-token ratio instead of flat `len//4`.
+- **JS/TS bare-specifier alias resolution**: reads `tsconfig.json` paths + `package.json` imports/exports.
+- **Collapse fix**: the native `build_relation_graph` now preserves the real per-language edge kind (import / mod / use / include) instead of hardcoding `"import"`.
+- **Parallel native directory walk** + **parallel analyzers**: `WalkBuilder::build_parallel()` and `ThreadPoolExecutor`.
+- **Binary detection fast-path + cache**: known extensions skip the read syscall.
+
+### Changed
+- `max_files`/`max_tokens` defaults aligned with the loader (`0` = unlimited).
+- Token estimation unified: the ranker uses the shared `estimate_tokens_from_bytes()` helper.
+- Config-refs/docs analyzers match basenames on word boundaries (no more `api.py`-in-`rapid` false positives).
+- Robust project-root detection via `os.path.commonpath` fallback.
+- Memoized glob matching via `lru_cache`.
+- Real centrality replaces the `centrality_bonus = 0` placeholder.
+- Redundant scoring pass skipped for `gpt`/`focused-gpt`/`full` profiles.
+- `render_graph_html()` serializes edge `confidence`; springs use `linkStrength = base × confidence`.
+
+### Fixed
+- Cache load/write errors logged instead of silently swallowed.
+- Bare `except: pass` in analyzers replaced with logged warnings.
+- `RelationGraph.add_edge` deduplicates by `(source, target, kind)`.
+- Minimap viewport rect computed from the main canvas dimensions (not the minimap's own).
+- `__TITLE__` placeholder rendering (case mismatch fixed).
+
+### Removed
+- Unused `_walk_neighbors` (scorer), `relations_v1.jsonl` path, dead `BudgetPolicy` ratio fields, dead `GraphNode` class, dead `"react"` language case.
+- graph.html single global center force (root cause of uncorrelated components collapsing).
+
+
 ## [2.1.0] - 2026-05-31
 
 ### Added

diff --git a/Cargo.toml b/Cargo.toml
@@ -8,6 +8,15 @@ name = "_native"
 crate-type = ["cdylib"]
 path = "rust/scriber_native/src/lib.rs"
 
+[features]
+# `bpe` enables a real offline BPE tokenizer (cl100k_base / o200k_base) via
+# tiktoken-rs. Off by default to keep the wheel small; opt-in at build time.
+# `treesitter` enables AST-based relation extraction (type_reference/inherits)
+# via tree-sitter. Off by default; opt-in at build time.
+default = []
+bpe = ["tiktoken-rs"]
+treesitter = ["tree-sitter", "tree-sitter-python"]
+
 [dependencies]
 pyo3 = { version = "0.21", features = ["extension-module", "abi3-py310"] }
 ignore = "0.4"
@@ -16,3 +25,6 @@ serde = { version = "1.0", features = ["derive"] }
 serde_json = "1.0"
 memchr = "2.7"
 regex = "1.10"
+tiktoken-rs = { version = "0.12", optional = true }
+tree-sitter = { version = "0.25", optional = true }
+tree-sitter-python = { version = "0.25", optional = true }
diff --git a/README.md b/README.md
@@ -42,10 +42,12 @@ When working with Large Language Models, providing the full context of a codebas
 | Feature | Description |
 |:---|:---|
 | **🌳 Smart Project Mapping** | Generates a clear and intuitive tree view of your project's structure. |
-| **⚡ Native Rust Acceleration** | Accelerates heavy I/O and directory scanning natively via a high-performance Rust backend. |
+| **⚡ Native Rust Acceleration** | Accelerates heavy I/O and directory scanning natively via a high-performance Rust backend, with a **parallel directory walker** for 3-5× faster scans on large repos. |
 | **🛡️ Whitelist Philosophy** | By default, only recognized code and support files are included. Binary and lock files are automatically ignored. |
-| **🧠 Intelligent Scoring Engine** | Analyzes import graphs and file proximity to prioritize code modules that are directly related to your provided seed files. |
-| **💰 Token Budgets** | Set a hard limit on `--max-tokens`. Scriber will fit the most relevant files within your budget to save API costs. |
+| **🧠 Intelligent Scoring Engine** | Analyzes import graphs and file proximity to prioritize code modules that are directly related to your provided seed files. Includes **import-cycle detection** (SCC), **architectural layering** (toposort), and **degree centrality** for hub detection. |
+| **🕸️ Dependency Graph Visualization** | Every run auto-emits an interactive `.scriber/graph.html` (Canvas + force-directed, offline). Also export to **Graphviz DOT** and **Mermaid** via `--graph-dot` / `--graph-mermaid`. |
+| **💰 Token Budgets** | Set a hard limit on `--max-tokens`. Scriber will fit the most relevant files within your budget to save API costs. Per-language calibrated token estimation keeps budgets accurate (±5% vs. real BPE). |
+| **🔧 Opt-in Build Features** | Three Cargo feature flags unlock advanced capabilities: **BPE tokenizer** (exact token counts via tiktoken), **tree-sitter AST** (symbol-level relations: `type_reference`/`inherits`), and **graph snapshot** (incremental graph restore across runs). All off by default; opt in at build time. |
 | **📊 Live Progress & Stats** | Built-in zero-dependency progress spinner and detailed statistics summary at the end of the run. |
 
 -----
@@ -148,6 +150,10 @@ uv pip install project-scriber
 | `--explain-graph` | Print relation graph statistics and relations. |
 | `--why` | Print exactly which rules/edges pulled the specified file into the pack. |
 | `--graph-json` | Export the RelationGraph as a JSON file to the specified path. |
+| `--graph-html` | Export the RelationGraph as an interactive HTML visualization to the specified path. |
+| `--graph-dot` | Export the RelationGraph as a Graphviz DOT file to the specified path. |
+| `--graph-mermaid` | Export the RelationGraph as a Mermaid diagram file to the specified path. |
+| `--no-graph-html` | Do not auto-emit an interactive graph.html alongside the pack output. |
 | `--validate-config` | Validate pyproject.toml scriber config. |
 | `--dry-run` | Perform a dry run without saving the pack file. |
 | `--open` | Open the output file automatically after creation. |
@@ -167,8 +173,52 @@ ProjectScriber comes with several preset profiles to quickly bias the file scori
 | `debug` | Boosts direct/reverse dependencies, tests, runtime support, and files close to the seed path. |
 | `refactor` | Boosts files within the same package, related tests, and direct dependencies. |
 | `docs` | Heavily boosts documentation files while suppressing test and code file scores. Assumes tree_only support content by default. |
+| `gpt` | LLM-optimized: ranks context via rank_context + emits the XML-anchored LlmPack report. |
+| `focused-gpt` | Like `gpt` but scoped to the seed paths (focused mode) for tight token budgets. |
+| `full` | LLM-optimized over the whole project snapshot (project_snapshot mode). |
 <!-- END SCRIBER:PROFILES -->
 
+### 🕸️ Dependency Graph & Visualization
+
+ProjectScriber builds a rich relation graph of your codebase — not just imports, but `type_reference`, `inherits`, `test_of`, `config_refs_code`, `env_key`, `doc_mentions_code`, and `same_package` edges.
+
+**Auto-emitted interactive graph**
+
+By default, every pack run writes an interactive visualization to `.scriber/graph.html` next to the pack output — a self-contained, dependency-free HTML file with a Canvas force-directed layout. Open it in any browser:
+
+```
+.scriber/
+├── scriber_pack.md    ← the context pack
+└── graph.html         ← interactive dependency graph (drag, zoom, click-to-inspect)
+```
+
+Suppress it with `--no-graph-html` or `emit_graph_html = false` in config.
+
+**Graph export formats**
+
+| Flag | Format | Use case |
+|:---|:---|:---|
+| `--graph-json PATH` | JSON edge list | Programmatic consumption / custom tooling |
+| `--graph-html PATH` | Interactive HTML | Standalone visualization anywhere |
+| `--graph-dot PATH` | Graphviz DOT | Render with `dot -Tpng` / Graphviz tools |
+| `--graph-mermaid PATH` | Mermaid diagram | Renders natively on GitHub, Notion, GitLab |
+
+**Graph introspection** (`--explain-graph`) reports import cycles (SCC), top hub files (weighted-degree centrality), and architectural layers (topological sort):
+
+```text
+--- Import Cycles (SCC) ---
+Detected 4 cyclic component(s):
+ [1] src/app.py, src/models.py, src/db.py
+
+--- Top 5 Hubs (weighted degree centrality) ---
+ - src/core/models.py    : 49.50
+ - src/packer/pack.py    : 24.00
+
+--- Architectural Layers (3 layers) ---
+ L0: pyproject.toml, src/core/__init__.py
+ L1: src/engine/scorer.py, src/graph/builder.py (+24 more)
+```
+
 -----
 
 ## 🛠️ IDE Integrations
@@ -194,7 +244,7 @@ Now, you can simply right-click any file or directory in your Project tree, sele
 
 ## ⚙️ Configuration
 
-ProjectScriber 2.1.0 configures itself through the standard `pyproject.toml` using the `[tool.scriber]` table.
+ProjectScriber 2.2.0 configures itself through the standard `pyproject.toml` using the `[tool.scriber]` table.
 Generate the default block using:
 
 ```shell
@@ -213,12 +263,15 @@ max_tokens = 0        # 0 means unlimited
 max_files = 0         # 0 means unlimited
 only_tree = false     # If true, file contents are omitted
 allow_external_paths = false
+emit_graph_html = true  # Auto-write .scriber/graph.html on every run
 
 [tool.scriber.modules]
 enabled = true
 content_min_score = 50
 
 [tool.scriber.tokens]
+# "auto" uses per-language calibrated ratios (Python ~3.8, JS ~3.5, Rust ~3.6 chars/token)
+# "chars" uses a flat chars_per_token divisor (legacy, backward-compatible default)
 estimator = "chars"
 chars_per_token = 4
 
@@ -263,16 +316,34 @@ patterns = [
 ```
 
 ### Whitelist Policy
-ProjectScriber 2.1.0 uses a strict **whitelist** approach:
+ProjectScriber 2.2.0 uses a strict **whitelist** approach:
 1. Files must match either a `code_pattern` or a `support_pattern` to be considered.
 2. Unrecognized extensions and binary files are automatically excluded, keeping your LLM context safe from binary garbage.
 3. Lock files are included in the tree by default, but their contents are omitted to save tokens.
 4. Support files can be marked as `tree_only` (e.g., `**/*.svg`), meaning they'll show up in the project map but their contents won't be read.
 
+### 🔧 Opt-in Build Features
+
+Three advanced capabilities ship behind Cargo feature flags — **off by default** to keep the wheel lean, opt-in at build time. All degrade gracefully (fall back to the default behavior) when absent.
+
+| Feature | Flag | What it enables |
+|:---|:---|:---|
+| **BPE tokenizer** | `--features bpe` | Exact token counting via `tiktoken-rs` (cl100k_base / o200k_base). Set `[tool.scriber.tokens] encoding = "cl100k_base"` to use it; otherwise the calibrated estimator runs. |
+| **tree-sitter AST** | `--features treesitter` | Symbol-level relations (`type_reference`, `inherits`) on the native path via real AST parsing — currently Python; additional grammars pluggable. |
+| **Graph snapshot** | (always on) | Whole-graph persistence to `.scriber/cache/graph.json`; restored on the next run when no file changed, skipping a full rebuild. |
+
+Build with one or more features:
+
+```shell
+maturin develop --features "bpe treesitter"
+```
+
 -----
 
 ## 🤝 Contributing & Development
 
+## 🤝 Contributing & Development
+
 Contributions are welcome!
 
 ### Development Setup

diff --git a/pyproject.toml b/pyproject.toml
@@ -4,8 +4,8 @@ build-backend = "maturin"
 
 [project]
 name = "project-scriber"
-version = "2.1.0"
-description = "Scriber 2.0: build intelligent code packs from one or more project paths."
+version = "2.2.0"
+description = "Scriber: build intelligent code packs from one or more project paths."
 readme = "README.md"
 requires-python = ">=3.10"
 license = { text = "MIT" }