title	NanoRAG
emoji	📚
colorFrom	indigo
colorTo	blue
sdk	gradio
sdk_version	4.19.2
app_file	app.py
pinned	false
license	mit
short_description	Retrieval-Augmented Generation for PDFs on Free CPU
tags	nlp rag pdf-qa faiss transformers

NanoRAG (Retrieval-Augmented Generation)

A lightweight, memory-efficient Question Answering system for PDF documents. Built to run entirely on free CPU tiers (e.g., Hugging Face Spaces free tier) without crashing.

Overview

This project demonstrates how to build a functional RAG pipeline under strict resource constraints. It allows users to upload a PDF and ask questions about its content in natural language. The system retrieves relevant text chunks and generates an answer using a Flan-T5 model.

Key Features

• Semantic Search: Uses SentenceTransformers + FAISS to find the most relevant document sections.
• Memory Optimization: Processes PDFs in streams and uses aggressive garbage collection to fit within 2GB RAM.
• Streaming UI: Real-time answer generation using TextIteratorStreamer.
• Zero-Cost Deployment: Designed specifically for CPU-only environments.

Tech Stack

• Framework: Gradio for the web interface.
• Vector Search: FAISS (CPU version).
• Embeddings: sentence-transformers/all-MiniLM-L6-v2 (small, fast, effective).
• LLM: google/flan-t5-small (encoder-decoder model good for instruction following).
• PDF Parsing: PyMuPDF (fitz).

Architecture

1. Ingestion: The PDF is read page-by-page to minimize memory footprint.
2. Chunking: Text is split into overlapping 384-character windows.
3. Embedding: Chunks are vectorized using MiniLM and stored in a FAISS IndexFlatIP index.
4. Retrieval: User queries are embedded and compared against the index (Cosine Similarity).
5. Generation: Top-k chunks are fed into Flan-T5 as context to generate the final answer.

How to Run Locally

1. Clone the repository

   git clone https://github.com/your-username/pdf-rag-hf.git
   cd pdf-rag-hf

2. Install dependencies

   pip install -r requirements.txt

3. Run the app

   python app.py

   Open http://127.0.0.1:7860 in your browser.

Example Questions

Once you've indexed a document, try asking:

• "What is the main conclusion of this paper?"
• "List the key arguments mentioned in the introduction."
• "Summarize the methodology used."

Limitations

• Model Size: Uses flan-t5-small (80M params) to ensure speed on CPU. Answers may be less nuanced than larger models (e.g., GPT-4).
• Concurrency: Single-threaded processing is enforced to prevent CPU throttling on free cloud instances.

License

MIT