| title | ResearchPath |
|---|---|
| emoji | 🗺️ |
| colorFrom | blue |
| colorTo | indigo |
| sdk | docker |
| pinned | false |
| license | mit |
| short_description | Agentic RL reading-path planner with grounded Q&A |
An agentic research-onboarding companion. Give it a target paper and your background; it builds a personalized, dependency-ordered reading plan with grounded explanations.
Demo domain: Reinforcement Learning. Architecture: domain-agnostic.
Getting into a new research field is brutal. You open the SOTA paper, it assumes 8 prior concepts. You read those papers, they assume 5 more. Existing tools (Perplexity, Elicit, Consensus) retrieve papers but don't plan — they don't tell you what order to read things in based on your specific background.
Input:
- A target paper or topic (e.g., "PPO")
- Your current background (e.g., "basic supervised ML, calculus, no RL")
Output:
- Prerequisite reading plan — a topologically-sorted reading list, with "why this is next" rationale. Optionally AI-annotated: each step gets a grounded 2-3 sentence explanation citing the actual paper.
- Concept genealogy — search any concept (e.g. "experience replay", "entropy") and trace how it appears across papers, textbooks, and course notes, sorted chronologically.
- Grounded Q&A — ask follow-ups, every claim cites
[source_id, p<page>]from a ~7k-chunk corpus spanning papers + textbooks + courses + tutorials. - (Stretch) Open problems surfacer — clusters "Future Work" sections from recent papers
The reading-path builder is a real planning problem:
read(target_paper) → extract assumed prerequisites
for each prerequisite:
if user_knows(prereq): skip
else: retrieve canonical paper for prereq → recurse
build dependency DAG → topologically sort → generate bridge explanations
Graph traversal + recursive retrieval + reasoning over user state. Not "embed query, return top-5 chunks."
Eval is the differentiator. Every change ships with numbers.
Gold dataset: 39 hand-authored (question, expected source, key claim) triples. 30 questions cover 10 canonical RL papers (difficulty-stratified: 7 easy / 15 medium / 8 hard); 9 new questions target textbook/tutorial sources (Sutton & Barto, RLHF Book, Silver lectures, Spinning Up) to measure whether the corpus expansion actually improves retrieval on foundational concepts.
| Metric | Baseline RAG | + Hybrid Retrieval |
|---|---|---|
| Retrieval Recall@5 | 90.0% | 89.7% |
| Citation Presence | 86.7% | 86.2% |
| Answer Correctness | 36.7% | 72.4% |
| Avg Latency (s) | 5.07 | 4.58 |
| RAG Tokens (in/out) | 33,498 / 7,000 | 31,439 / 6,905 |
Hybrid BM25+FAISS via RRF fusion (n=29). Answer Correctness nearly doubled (+35.7 pp) with no latency regression — driven by hybrid correctly surfacing in-paper chunks that dense embeddings de-prioritized.
Corpus expanded to all 17 canonical RL papers (PER, PPO, SAC, IMPALA, MuZero, DreamerV3, Decision Transformer added). Prerequisite graph updated with 11 new edges.
| Metric | Baseline RAG | + Hybrid Retrieval | + Reranker |
|---|---|---|---|
| Retrieval Recall@5 | 84.0% | pending | 83.3% |
| Citation Presence | 80.0% | pending | 83.3% |
| Answer Correctness | 48.0% | pending | 56.7% |
| Avg Latency (s) | 4.63 | pending | 5.13 |
| RAG Tokens (in/out) | 27,027 / 6,336 | pending | 31,579 / 11,215 |
v2 baseline n=25 (5 skipped, Groq 100k TPD hit). Reranker: BM25+FAISS+CrossEncoder, n=30. Hybrid v2 pending token reset. Reranker adds +8.7 pp over v2 baseline; larger corpus raises baseline from 37% → 48% even without retrieval improvements.
Corpus expanded beyond papers to include textbooks, course slides, and tutorials. Gold dataset expanded to 39 questions to measure textbook/tutorial retrieval quality.
| Source | Chunks | Type |
|---|---|---|
| 19 arXiv papers (17 original + 2 surveys) | ~2,000 | paper |
| Sutton & Barto RL textbook (2nd ed.) | 2,446 | textbook |
| RLHF Book | 938 | textbook |
| Stanford CS224R notes | 153 | course |
| David Silver UCL/DeepMind (10 lectures) | ~482 | course |
| 7 web tutorials (Spinning Up, Lilian Weng ×4, HF blog) | ~260 | tutorial |
v3 hybrid eval (in progress, n=28 across two batches, Gemini judge, 7,093-chunk index):
| Metric | v3 Hybrid (n=28, partial) |
|---|---|
| Retrieval Recall@5 | 75.0% |
| Citation Presence | 71.4% |
| Answer Correctness | 64.3% |
| Avg Latency (s) | ~10 |
Two-batch eval split across the Gemini 20-req/day free tier ceiling. Batch 1 covered value-based + RLHF papers (dqn/ddqn/dueling/rainbow/instructgpt/dpo + cross, n=18, Recall 72.2% / Citation 77.8% / Correctness 66.7%). Batch 2 covered policy-gradient + actor-critic papers (trpo/gae/a3c/ddpg, n=10, Recall 80.0% / Citation 60.0% / Correctness 60.0%). Combined aggregate shown above. Remaining 11 questions (textbook/tutorial sources + sb_/mdp_/pg_thm/rlhf/exploration) will be added over the next 1-2 days as quota refreshes.
| Layer | Choice | Why |
|---|---|---|
| Planning LLM | Gemini 2.5 Flash Lite (free tier) | Strong reasoning, generous free quota |
| Fast LLM | Groq Llama 3.3 70B (free) | Fast inference for inner-loop retrieval |
| Embeddings | BAAI/bge-small-en-v1.5 | CPU-friendly, strong on academic text, free |
| Vector store | FAISS IndexFlatIP | Local, free, exact cosine, fast at ~7k chunks |
| Retrieval | BM25 + FAISS via RRF + CrossEncoder rerank | Three tiers, each measurably better |
| Agent framework | LangGraph state machine (supervisor + ReAct + critic) | Typed state, conditional routing, hierarchical delegation |
| Reading planner | Static DAG + BFS + Kahn's topo sort | Deterministic planning, no LLM cost |
| Observability | Custom metrics + tracing modules, LangSmith mirror | Per-request token/cost/latency, granular event timeline |
| Tool exposure | MCP server (FastMCP) | Standard agentic-tool protocol; Claude Desktop can call our corpus |
| Auth | OAuth (HF Spaces native or Google OIDC) | Gates paid API calls behind a real identity |
| UI | Streamlit | Demo-grade, ships fast |
| Eval | Custom harness + LLM-as-judge | Citation recall, answer correctness, latency |
| Deploy | Hugging Face Spaces | Free public URL |
┌──────────────────────┐
│ Supervisor │ rule-based → LLM fallback router
│ (intent classifier) │
└──────────┬────────────┘
│
┌─────────────────┼──────────────────┐
▼ ▼ ▼
┌─────────┐ ┌──────────────┐ ┌─────────────┐
│ Plan │ │ Research │ │ Concept │
│ agent │ │ agent (ReAct│ │ agent │
│ │ │ ≤3 iters) │ │ │
└────┬────┘ └──────┬───────┘ └──────┬──────┘
│ tool: plan_path │ tools: │ tool:
│ + annotate │ search_corpus │ trace_concept
│ │ │
└──────────────────┴──────────────────┘
▼
┌──────────────┐
│ Critic │ mechanical citation check
│ (verifies │ vs retrieved chunks
│ citations) │
└──────┬────────┘
▼
Answer + metrics + trace
Every node reads/writes a typed AgentState. Every LLM call is recorded
to a per-request MetricsCollector (tokens, latency, cost) and Tracer
(event timeline). Optional LangSmith mirror for shareable traces.
Entry point: researchpath.agents.run_agent(query, intent="research").
- Week 1 — Repo scaffold + smoke test
- Week 1 — arXiv corpus ingestion (10/17 RL papers, 1,093 chunks)
- Week 1 — Baseline RAG (FAISS + bge-small + Gemini/Groq), smoke tested
- Week 2 — Eval harness + 30-question gold dataset
- Week 2 — Baseline RAG numbers (Recall@5 90%, Answer Correctness 37%)
- Week 2 — Hybrid BM25+FAISS retrieval via RRF (Answer Correctness 72%, +35 pp)
- Week 3 — Reranker (cross-encoder): +8.7 pp over v2 baseline
- Week 3 — Agentic planning loop: offline prerequisite-chain planner, 7 tests passing
- Week 3 — Full 17-paper corpus (1,789 chunks) + expanded prerequisite DAG (21 edges)
- Week 4 — Streamlit UI + HF Spaces deploy (Docker)
- Week 4 — Tier 1 corpus expansion: Sutton & Barto, RLHF Book, CS224R, 5 web tutorials → 5,531 chunks
- Week 4 — Tier 2+3: David Silver 10-lecture course + 2 survey papers + 2 more Weng posts → 7,093 chunks
- Week 4 — LLM-powered plan annotations:
parse_background(NL→known IDs) +annotate_plan(RAG-grounded per-step explanations) - Week 4 — Concept Genealogy tab: trace any RL concept across the full corpus, sorted chronologically
- Week 4 — Gold dataset expanded to 39 questions (9 new textbook/tutorial Qs); 13 annotator tests
- Week 5 — Multi-agent upgrade: LangGraph state machine (supervisor + ReAct research agent + critic), hierarchical delegation
- Week 5 — Production observability: per-request token/cost/latency metrics, granular event tracing, LangSmith mirror, Metrics & Trace tab in UI
- Week 5 — MCP server: FastMCP server exposing
search_corpus,plan_path,trace_conceptso Claude Desktop and other MCP clients can use the corpus - Week 5 — OAuth: pluggable auth (HF Spaces native or Google OIDC); ALLOWED_EMAILS allowlist; no-op for local dev
- Week 5 — 33 new tests for metrics, tracing, and agent layer (53 tests total passing)
- Week 4 — Hybrid v3 eval (running), ablation table complete, demo video
# 1. Install uv (one-time): https://astral.sh/uv
# 2. Sync dependencies
uv sync
# 3. Set up env
copy .env.example .env
# Fill in GEMINI_API_KEY and GROQ_API_KEY in .env
# 4. Run smoke test
uv run python scripts/smoke_test.py
# 5. Build corpus — three sources (one-time, ~25 min total)
# 5a. Research papers (17 arXiv PDFs)
uv run python scripts/fetch_corpus.py
# 5b. Textbooks + course notes (Sutton & Barto, RLHF Book, CS224R)
uv run python scripts/fetch_pdfs.py
# 5c. Web tutorials (Spinning Up, Lilian Weng, HF blog)
uv run python scripts/fetch_web_sources.py
# 5d. Parse all sources → embed → FAISS index (~7k chunks)
uv run python scripts/parse_corpus.py
uv run python scripts/build_index.py
# 6. Ask a question
uv run python scripts/ask.py "What is the key idea behind PPO?"
uv run python scripts/ask.py --hybrid --rerank "How does Rainbow combine Double DQN and PER?"
# 7. Get a reading plan
uv run python scripts/plan.py --target PPO
uv run python scripts/plan.py --target Rainbow --known DQN
# With AI-grounded per-step explanations:
uv run python scripts/plan.py --target PPO --background "I know supervised ML and basic calculus, no RL" --annotate
# 8. Trace a concept across the corpus
# (via Streamlit Tab 3, or add a genealogy CLI in a future iteration)
# 9. Run the full eval
uv run python scripts/run_eval.py --hybrid --gemini-judge
uv run python scripts/run_eval.py --hybrid --gemini-judge --resume # resume after rate limit