Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

sqlite-rag

Minimal RAG: Ollama embeddings + cosine similarity in SQLite. No vector database. ~150 lines, standard library only.

sqlite-rag demo

Most RAG stacks pull in a vector database (Chroma, Pinecone, Qdrant, FAISS), a web framework, and a dozen dependencies. For small-to-medium corpora you don't need any of that. This is the whole thing: embeddings come from a local Ollama server, vectors and text live in one SQLite file, and search is plain cosine similarity in Python.

Why

  • Zero dependencies — pure Python standard library (sqlite3, urllib, json, math).
  • One file, readable in one sitting — easy to audit, fork, and embed in another project.
  • Local & private — nothing leaves your machine.
  • Portable index — the whole knowledge base is a single .db file you can copy around.

Quickstart

ollama pull nomic-embed-text        # embedding model
python example.py

Example run (real output)

Query: How far from Earth does the telescope orbit, and at which point?
  [0.807] jwst: The James Webb Space Telescope launched on 25 December 2021 ...

Generated answer:
  The telescope orbits the Sun at the second Lagrange point (L2),
  about 1.5 million kilometres from Earth.

Usage

from sqlite_rag import RAG

rag = RAG("mydocs.db")
rag.ingest_file("notes.md")                 # or rag.ingest("some text", source="x")
for hit in rag.search("how do I reset it?", k=4):
    print(hit["score"], hit["source"], hit["text"][:80])

How it works

  1. Ingest — text is split into overlapping chunks; each chunk is embedded via Ollama's /api/embeddings and stored in SQLite together with its vector (as JSON).
  2. Search — the query is embedded and compared against every stored vector with cosine similarity, computed in Python; the top-k chunks are returned.

That's it. No index structures, no ANN — a linear scan is plenty for thousands of chunks and keeps the code trivial to understand.

nomic-embed-text detail: the model expects task prefixes. This library adds search_document: to stored chunks and search_query: to queries automatically — forgetting these is a common cause of poor recall.

Configuration (env vars)

Variable Default
OLLAMA_URL http://127.0.0.1:11434
EMBED_MODEL nomic-embed-text
CHAT_MODEL (example only) qwen2.5:7b

Limits (honest)

  • Linear scan: fine up to ~tens of thousands of chunks, then add an ANN index.
  • Naive fixed-size chunking — swap in your own splitter if structure matters.
  • No re-ranking; add a cross-encoder if you need higher precision.

License

MIT


See also

Part of a small collection of local-first AI and ESP32 / maker tools:

⭐ If this saved you time, a star helps others find it.

Responsible use & EU AI Act · Uso responsabile · Uso responsable · Usage responsable

EN — sqlite-rag is a free, open-source developer tool/library (MIT), not an end-user AI system, and it does not bundle any AI model — you connect your own (e.g. Ollama embeddings and an LLM of your choice). Outputs from any model may be inaccurate or biased; verify important results (not legal, medical or financial advice). Third-party components (e.g. Ollama embeddings and an LLM of your choice) keep their own licenses and usage policies. Under the EU AI Act (Reg. (EU) 2024/1689), transparency and other obligations apply to the product/deployer built with this tool, not to this library in isolation; as a free & open-source component it falls under the Act's open-source provisions.

IT — sqlite-rag è uno strumento/libreria per sviluppatori libero e open source (MIT), non un sistema di IA per l'utente finale, e non include alcun modello di IA (usi il tuo). Gli output dei modelli possono essere errati o distorti: verifica i risultati importanti (non è consulenza legale, medica o finanziaria). I componenti di terzi mantengono le proprie licenze. Ai sensi dell'AI Act (Reg. UE 2024/1689), gli obblighi di trasparenza ricadono sul prodotto/deployer costruito con questo strumento, non sulla libreria in sé; come componente libero e open source rientra nelle relative esenzioni.

ES — sqlite-rag es una herramienta/biblioteca para desarrolladores libre y de código abierto (MIT), no un sistema de IA para el usuario final, y no incluye ningún modelo de IA (conectas el tuyo). Las salidas de los modelos pueden ser inexactas o sesgadas: verifica los resultados importantes (no es asesoramiento legal, médico ni financiero). Los componentes de terceros conservan sus licencias. Según el Reglamento de IA (UE 2024/1689), las obligaciones de transparencia recaen en el producto/implementador, no en la biblioteca en sí; como componente libre y de código abierto se acoge a las disposiciones open source.

FR — sqlite-rag est un outil/bibliothèque pour développeurs libre et open source (MIT), pas un système d'IA destiné à l'utilisateur final, et n'inclut aucun modèle d'IA (vous connectez le vôtre). Les sorties des modèles peuvent être inexactes ou biaisées : vérifiez les résultats importants (ce n'est pas un conseil juridique, médical ou financier). Les composants tiers conservent leurs licences. Selon le Règlement IA (UE 2024/1689), les obligations de transparence incombent au produit/déployeur, pas à la bibliothèque elle-même ; en tant que composant libre et open source, il relève des dispositions open source.

Free course: AI Basics for Developers

Article 4 of the EU AI Act requires anyone building or using AI professionally to have documented AI competence — no certificate is legally required, just a record. We built a free, voluntary 25-minute course that gives you that record, with a certificate of attendance: https://developcontroller.com/corso/ No sign-up, no tracking, available in EN / IT / ES / FR.

About

Minimal RAG: Ollama embeddings + cosine similarity in SQLite. No vector DB. Pure Python stdlib.

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages