A modular AI-powered healthcare assistant built using LLMs + LangChain, designed to provide structured, personalized insights based on disease detection and patient data.
This project is not a general chatbot. It is a multi-agent system that:
- Takes structured medical JSON input
- Uses specialized AI agents for different tasks
- Produces schema-constrained outputs
- Integrates ML disease detection + LLM reasoning
User Input (Medical JSON / Image)
↓
Disease Detection Models
↓
Structured JSON Output
↓
LangChain Agent Router
├── Lifestyle Suggestor Agent
├── Hospital Finder Agent (with external API)
├── Cost Estimator Agent
└── Diet Planner Agent
↓
Response Aggregator
↓
Final Structured Output
- Generates personalized lifestyle recommendations
- Focus: habits, sleep, exercise, stress
- Finds relevant hospitals based on disease & location
- Can integrate with APIs (Google Maps / SerpAPI)
- Provides approximate treatment cost (INR)
- Includes cost breakdown and influencing factors
- Generates disease-specific diet plans
- Supports Indian food context and dietary preferences
The system integrates pretrained deep learning models for automated disease prediction:
- Model: AurevinP/pneumonia-classifier-effnetb0
- Architecture: EfficientNet-B0
- Link: https://huggingface.co/AurevinP/pneumonia-classifier-effnetb0
- Model: rafalosa/diabetic-retinopathy-224-procnorm-vit
- Architecture: Vision Transformer (ViT)
- Link: https://huggingface.co/rafalosa/diabetic-retinopathy-224-procnorm-vit/tree/main
- Model: subx24/ml-alzheimer-models
- Architecture: VGG19
- Link: https://huggingface.co/subx24/ml-alzheimer-models/tree/main
- LLM: Gemini API
- Framework: LangChain
- Backend: Node.js / Python (extensible)
- ML Models: HuggingFace Transformers
- Data Format: Structured JSON
All agents consume a standardized JSON input:
{
"patient": {
"age": 45,
"gender": "male"
},
"disease": {
"name": "Type 2 Diabetes",
"severity": "moderate"
},
"location": {
"city": "Chandigarh"
},
"preferences": {
"diet_type": "vegetarian",
"budget": "medium"
}
}- Node.js
- Gemini API Key
-
Install dependencies:
npm install
-
Set your API key in
.env.local:GEMINI_API_KEY=your_api_key_here -
Run the app:
npm run dev
- ✅ Multi-agent architecture (not a chatbot)
- ✅ Structured medical reasoning
- ✅ Modular and scalable design
- ✅ Integration of ML + LLM
- ✅ Real-world healthcare use-case
| Feature | ChatGPT | This System |
|---|---|---|
| Input Type | Unstructured text | Structured JSON |
| Output | Free-form | Schema-constrained |
| Architecture | Single model | Multi-agent system |
| External Tools | Limited | Integrated APIs |
| Domain Focus | General | Healthcare-specific |
🔮 Future Improvements
Real-time hospital API integration
Patient history tracking
Risk prediction models
Doctor recommendation ranking system
📌 Author Developed as part of an AI healthcare system project using modern LLM orchestration techniques.