Metatron

Multi-project web feed manager with cross-outlet deduplication.

You point Metatron at sources, organized by project. It polls them, dedups the articles (URL canonicalization -> normalized title -> token overlap -> LLM tiebreaker), and serves you the unique articles via a small HTTP API. RSS is the source it speaks today; the architecture is medium-agnostic and grows to other web sources over time.

It does this one thing. It doesn't curate, it doesn't summarize, it doesn't have opinions about what's interesting. That's for whatever calls it.

Features

• Project-scoped feed collections, so unrelated reading workflows stay separate.
• Per-feed categories, copied onto fetched articles for filtered reading.
• Deduplicated article lists that keep one canonical article per story while preserving alternate sources on the detail endpoint.
• Cheap deterministic duplicate checks before the optional Claude CLI tiebreaker runs.
• SQLite storage with no external service dependency.
• Small bearer-authenticated FastAPI surface for callers and dashboards.

Install

The CLI is metatron. The published package is metatron-cli (the bare metatron name was already taken on PyPI):

pipx install metatron-cli                                   # from PyPI
pipx install git+https://github.com/mattweberio/metatron    # from GitHub (latest main)

Python 3.12 or newer is required.

Configure

metatron config init       # writes ~/.config/metatron/config.toml
metatron config show       # prints the resolved config

Edit ~/.config/metatron/config.toml. The optional LLM tiebreaker is used to catch "same story, different outlet" duplicates after cheap URL/title/token checks. It shells out to the local claude CLI, so it uses your existing Claude CLI authentication rather than an API key in the Metatron config:

[llm]
enabled = true
model = "sonnet"
binary = "claude"

[api]
api_token = "your-bearer-token"

With [llm].enabled = false, dedup still uses canonical URL, normalized title, and token-overlap shortlisting. The Claude tiebreaker is what catches harder "same story, different headline" duplicates across outlets.

Run

metatron serve             # HTTP API on 127.0.0.1:8765

Use

Bearer-token auth on everything except /health.

TOKEN="your-bearer-token"
BASE="http://127.0.0.1:8765"

# Create a project
curl -s -X POST "$BASE/projects" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"name": "ai-news"}'

# Add feeds. Categories are optional but useful for filtered reading.
PROJECT_ID=...
curl -s -X POST "$BASE/projects/$PROJECT_ID/feeds" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://hnrss.org/frontpage", "name": "Hacker News", "category": "tech"}'

# List deduplicated articles
curl -s -H "Authorization: Bearer $TOKEN" \
  "$BASE/projects/$PROJECT_ID/articles?limit=20"

# List only articles from feeds in given categories (comma-separated; omit for all)
curl -s -H "Authorization: Bearer $TOKEN" \
  "$BASE/projects/$PROJECT_ID/articles?category=writing,ideas&limit=20"

# Force a synchronous refresh
curl -s -X POST -H "Authorization: Bearer $TOKEN" \
  "$BASE/projects/$PROJECT_ID/refresh"

Bulk feed import

metatron seed-feeds my-project ./feeds.json

Where feeds.json is:

{
  "feeds": [
    { "url": "https://hnrss.org/frontpage", "name": "Hacker News", "category": "tech" },
    { "url": "https://www.theverge.com/rss/index.xml", "name": "The Verge", "category": "tech" }
  ]
}

Categories

Feeds can carry a free-form category such as tech, ai, writing, or ideas. When Metatron stores an article, it copies the feed category onto the article row. That makes category filtering stable even if you later rename the feed.

# Add a categorized feed
curl -s -X POST "$BASE/projects/$PROJECT_ID/feeds" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/rss.xml", "name": "Example", "category": "ideas"}'

# Read only one category
curl -s -H "Authorization: Bearer $TOKEN" \
  "$BASE/projects/$PROJECT_ID/articles?category=ideas"

# Read several categories
curl -s -H "Authorization: Bearer $TOKEN" \
  "$BASE/projects/$PROJECT_ID/articles?category=tech,ai,writing"

The filter is comma-separated and exact-match after trimming whitespace. Omit category to read all categories.

How dedup works

Cheapest-to-most-expensive, short-circuits on first match:

1. Canonical URL. Strip utm_*, fbclid, etc. Same canonical URL means exact duplicate, dropped.
2. Normalized title. Lowercase, drop punctuation, strip newsroom prefixes like BREAKING: and UPDATE -. Same normalized title joins the existing article's cluster.
3. Token overlap. Jaccard similarity on title and summary tokens. Scores at or above 0.12 shortlist candidates.
4. Claude tiebreaker. If enabled, Claude reads shortlisted pairs and decides whether they cover the same story.

Articles in a cluster are stored individually (you can see the alternative sources via GET /articles/{id}) but /projects/{id}/articles returns only the canonical from each cluster.

Storage

Single SQLite file at ~/.local/share/metatron/metatron.db. WAL mode, FK constraints on. Override with [database].path in config.

API endpoints

Method	Path	Description
GET	/health	Liveness; reports LLM enablement
POST	/projects	Create a project
GET	/projects	List projects
DELETE	/projects/{id}	Delete a project (cascades feeds + articles)
POST	/projects/{id}/feeds	Add a feed
GET	/projects/{id}/feeds	List feeds
DELETE	/feeds/{id}	Remove a feed
GET	/projects/{id}/articles?since=&category=&limit=	List deduped articles (category is a comma-separated filter; omit for all)
POST	/projects/{id}/refresh	Synchronously poll project's feeds
GET	/articles/{id}	Article detail + cluster members

Development

python3 -m venv .venv
. .venv/bin/activate
python -m pip install -U pip
python -m pip install -e '.[dev]'
python -m pytest -q

Tests cover normalization, the DB layer, the dedup pipeline (with a fake LLM judge), the feed poller (with mocked RSS), and the API surface.

Release instructions live in RELEASING.md. Public repo contribution and security guidance live in CONTRIBUTING.md and SECURITY.md.

Package metadata

Field	Value
PyPI distribution	metatron-cli
Installed command	metatron
Source	https://github.com/mattweberio/metatron
Issues	https://github.com/mattweberio/metatron/issues
License	MIT

License

MIT. See LICENSE.