RAG Chatbot using FastAPI + LangChain + Ollama (LLM)

A privacy-first, fully local AI chatbot built with FastAPI and LangChain that runs entirely on your own infrastructure — no cloud LLM costs, no data leaving your system.

HuggingFace embeddings power the semantic understanding of your documents, while Ollama serves open-source language models locally for response generation. PostgreSQL with pgvector acts as the vector store, supporting initialization from text and CSV data sources — making this system ideal for enterprise knowledge bases, compliance tools, and sensitive data environments.

Features

• Initialize the database with text or CSV data
• Perform chat queries using retrieval-augmented generation (RAG)
• Conversation history management

Prerequisites

• Python 3.10+
• PostgreSQL database with PGVector extension

  docker run --name pgvector-container -e POSTGRES_USER=langchain -e POSTGRES_PASSWORD=langchain -e POSTGRES_DB=langchain -p 5432:5432 -d pgvector/pgvector:pg16

• Ollama server for LLM

Installation

1. Clone the repository:

   git clone https://github.com/mrankitvish/rag-chatbot.git
   cd rag-chatbot

2. Create a virtual environment and activate it:

   python -m venv venv
   source venv/bin/activate  # On Windows use `venv\Scripts\activate`

3. Install the required packages:

   pip install -r requirements.txt

4. Create a .env file or rename .env_example in the root directory and set the following environment variables:

   EMBEDDING_MODEL=your_huggingface_embedding_model
   COLLECTION_NAME=your_pgvector_collection_name
   DB_NAME=your_database_name
   DB_IPADDR=your_database_ip
   DB_PORT=your_database_port
   DB_USER=your_database_user
   DB_PASSWD=your_database_password
   DATA=path_to_your_text_data_file
   DATA_CSV=path_to_your_csv_data_file
   LLM_MODEL=your_Ollama_model_name
   LLM_BASEURL=your_Ollama_server_url
   TOP_K=5

   Note Here LLM is used via Ollama

Running the Application

1. Start the FastAPI application:

   fastapi dev app.py --host 0.0.0.0

2. The application will be accessible at http://0.0.0.0:8000.

API Endpoints

Root Endpoint

• GET /

  Returns a greeting message.

  Response:

  {
      "message": "RAG ChatBot built for you to chat with your data."
  }

Database Initialization

• POST /db_init/{txt_or_csv}

  Initializes the database with text or CSV data.

  Path Parameters:

  • txt_or_csv: Specifies whether to initialize with text (txt) or CSV (csv) data.

  Responses:

  {
      "message": "Database initialized successfully"
  }

  {
      "message": "Invalid input"
  }

Chat Endpoint

• POST /chat

  Performs a chat query using the RAG chain.

  Request Body:

  {
      "question": "Your question here"
  }

  Response:

  {
      "answer": "Generated response based on the question and context."
  }

Example Usage

Initializing the Database

To initialize the database with text data:

curl -X POST http://127.0.0.1:8000/db_init/txt

To access SwaggerUI

http://127.0.0.1:8000/docs