Database

WikiNow uses SQLite with FTS5 full-text search. The database is a disposable cache over the markdown files -- delete wikinow.db and it rebuilds automatically on next access.

---

What Gets Tracked

The database tracks five things:

Articles

Every wiki page the AI writes gets indexed with its metadata:

path:        concepts/transformer.md
title:       Transformer Architecture
summary:     Self-attention mechanism for sequence modeling
confidence:  high
created:     2026-04-25
updated:     2026-04-27

This is what powers wn stats, wn search, list_all_articles(), and the lint checks.

Full-Text Search

Every article's title and body content (with frontmatter stripped) is indexed in an FTS5 virtual table with porter stemming.

What this means in practice:

Search "transformer"  --> matches "transformers", "Transformer"
Search "running"      --> matches "run", "runs", "running"
Search "machine learning" --> matches pages containing both words

Links

Every [[wikilink]] in every page is tracked. This enables:

• Dead link detection -- lint() finds [[wikilinks]] pointing to pages that don't exist
• Orphan detection -- lint() finds pages that nothing links to

Tags

Every tag from every page's frontmatter is tracked with counts. list_all_tags() returns something like:

ml:           5 articles
attention:    3 articles
architecture: 2 articles

Raw Sources

Every file in raw/ is tracked with its source URL, content hash, and compiled status. This enables:

• Dedup -- has_content_hash() checks if content was already ingested
• Pending detection -- lint() finds sources that haven't been compiled yet
• Stats -- get_project_stats() shows compiled vs pending counts

---

Self-Healing

The database syncs with the filesystem every time it opens:

• You add a .md file (e.g., via Obsidian) --> database indexes it automatically
• You edit a file --> database detects the newer timestamp and re-indexes
• You delete a file --> database removes the entry, its tags, its links, and its search index
• You delete wikinow.db --> everything rebuilds from the files on next access

No rebuild command. No migration scripts. The files are always truth.

---

Cascade Deletes

When an article is deleted from the database, its tags, links, and search entry are automatically removed via SQLite foreign key cascades. This prevents orphaned data from accumulating.

---

How Search Stays Safe

FTS5 has its own query syntax -- characters like *, ", (, ) and keywords like AND, OR, NEAR have special meaning. A naive search for C++ or an unmatched " would crash.

Every search query is wrapped in double quotes before reaching FTS5, which tells it to treat the input as a literal phrase. Internal quotes are escaped. If anything still fails, the search returns empty results instead of crashing.

---

Performance

• WAL mode -- Write-Ahead Logging for better concurrent read performance
• 5 indexes -- on confidence, link targets, tags, compiled status, content hash
• Point lookups -- dedup checks use an indexed content hash query
• No network -- everything is local SQLite, zero latency