RAG Learning Lab

An interactive local web app for exploring a Retrieval-Augmented Generation (RAG) pipeline end to end. Instead of hiding everything behind a chat window, it exposes the documents, chunks, embeddings, retrieval scores, assembled prompt, sources, latency and token cost.

Pipeline at a glance

PDF / DOCX / TXT / blog URL → text extraction → chunking → OpenAI embeddings → ChromaDB retrieval → prompt assembly → answer with sources

What you can inspect

Panel	What is visible
Documents	File upload or blog crawl, extracted text, folders and source metadata
Chunking Lab	Fixed-size, recursive and sentence-based chunking with chunk statistics
Embeddings	OpenAI embeddings projected to 2-D with PCA and cosine-similarity comparison
Chat	Retrieved chunks, relevance scores, source references, the full prompt and token counts
Metrics	Per-query latency, token usage and estimated API cost
Tutorial	A built-in walkthrough of the complete workflow

Built-in walkthrough

Stack

• Backend: FastAPI, ChromaDB, OpenAI SDK, Python
• Frontend: React 19, TypeScript, Vite, Tailwind CSS
• Quality checks: Pytest, ESLint, TypeScript and GitHub Actions

Run locally

Prerequisites

• Python 3.9+
• Node.js 18+
• An OpenAI API key for embeddings and chat

1. Clone

git clone https://github.com/KonP969/rag-learning-lab.git
cd rag-learning-lab

2. Start the backend

cd backend
python -m venv .venv

Activate the environment:

# Windows PowerShell
.\.venv\Scripts\Activate.ps1
Copy-Item .env.example .env

# macOS / Linux
source .venv/bin/activate
cp .env.example .env

Add your key to backend/.env, then run:

pip install -r requirements.txt
python -m uvicorn main:app --reload --port 8000

The API runs at http://localhost:8000; Swagger UI is available at http://localhost:8000/docs.

3. Start the frontend

cd frontend
npm ci
npm run dev

The app runs at http://localhost:5173.

Checks

# Backend
cd backend
python -m pytest tests -q

# Frontend
cd ../frontend
npm run lint
npm run build

The repository also contains a separate RAG evaluation suite in backend/tests/rag_evaluation/. It calls the running API and may use paid OpenAI requests, so it is not part of the default CI job.

Project structure

backend/
├── api/           # FastAPI routes
├── models/        # Pydantic models
├── services/      # Parsing, chunking, embeddings, retrieval and LLM logic
├── tests/         # Unit tests and the optional RAG evaluation suite
├── persistence.py # Local JSON persistence helpers
└── main.py

frontend/
└── src/
    ├── api/           # Typed API client
    ├── components/    # Documents, chunks, embeddings, chat, metrics and tutorial
    └── types/         # Shared TypeScript types

Current scope and limits

• Documents, chunks and folders are stored locally as JSON; embeddings persist in local ChromaDB storage.
• The app is intended for local, single-user learning and experimentation, not production deployment.
• The current “semantic” chunking option preserves sentence boundaries; it does not compare sentence embeddings.
• Public websites may rate-limit or block the blog crawler.
• API use may generate OpenAI charges. Keep keys in backend/.env; the file is ignored by Git.

AI-assisted development

Claude Code and Codex were used during iterative implementation, debugging, refactoring and verification of the web application.

License

MIT — see LICENSE.