Inferno - ML Model Serving Platform

A scalable backend service for hosting and serving machine learning models through REST APIs, built with Flask, RabbitMQ, MongoDB, and Docker.

Features

• REST API: Clean REST endpoints for model inference with Swagger documentation
• Asynchronous Processing: Handles concurrent requests using Celery with Redis
• Model Management: Easy loading and serving of ML models with versioning support
• Data Persistence: Stores model outputs and user queries in MongoDB
• Containerization: Fully containerized with Docker and Kubernetes support
• Security: API authentication, rate limiting, and input validation
• Monitoring: Built-in health checks, analytics, and comprehensive logging
• Scalability: Horizontal scaling support with multiple workers
• Caching: Redis-based caching for improved performance
• A/B Testing: Built-in support for model A/B testing and traffic splitting

Architecture

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   Flask API     │    │   Celery        │    │   MongoDB       │
│   (Port 5000)   │───▶│   Workers       │───▶│   (Port 27017)  │
└─────────────────┘    └─────────────────┘    └─────────────────┘
         │                       │
         ▼                       ▼
┌─────────────────┐    ┌─────────────────┐
│   Redis         │    │   Flower        │
│   (Port 6379)   │    │   (Port 5555)   │
└─────────────────┘    └─────────────────┘

Quick Start

Using Docker Compose (Recommended)

1. Clone and navigate to the project:

   cd /Users/ayush/Downloads/Inferno

2. Start all services:

   docker-compose up -d

3. Initialize sample models:

   docker-compose exec web python scripts/init_models.py

4. Test the API:

   python scripts/test_api.py

Manual Setup

1. Install dependencies:

   pip install -r requirements.txt

2. Start MongoDB and Redis:

   # MongoDB
   mongod --dbpath /path/to/your/db
   
   # Redis
   redis-server

3. Initialize sample models:

   python scripts/init_models.py

4. Start Celery worker (in a separate terminal):

   celery -A app.celery worker --loglevel=info

5. Start Flask application:

   python app.py

API Endpoints

Health Check

GET /health

Returns the health status of all services.

List Models

GET /models

Returns a list of available ML models.

Create Inference Request

POST /inference/{model_name}
Content-Type: application/json

{
  "features": [1.2, 2.3, 3.4, 4.5]
}

Check Inference Status

GET /inference/{query_id}/status

Get Inference Result

GET /inference/{query_id}/result

Analytics

GET /analytics/queries

Returns query statistics and model usage analytics.

Available Services

• API: http://localhost:5000
• API Documentation: http://localhost:5000/docs (Swagger UI)
• Health Check: http://localhost:5000/health
• Flower Dashboard: http://localhost:5555
• MongoDB: localhost:27017
• Redis: localhost:6379

Authentication

• API Key: demo-key-123 (for testing)
• Admin Key: admin-key-456 (for admin operations)

Example Usage

Python Client

import requests
import time

# Create inference request
response = requests.post('http://localhost:5000/inference/sample_classifier', 
                        json={'features': [1.2, 2.3, 3.4, 4.5]})
query_id = response.json()['query_id']

# Check status
time.sleep(2)
status = requests.get(f'http://localhost:5000/inference/{query_id}/status')
print(status.json())

# Get result
result = requests.get(f'http://localhost:5000/inference/{query_id}/result')
print(result.json())

cURL Examples

# Health check
curl http://localhost:5000/health

# List models
curl http://localhost:5000/models

# Create inference
curl -X POST http://localhost:5000/inference/sample_classifier \
  -H "Content-Type: application/json" \
  -d '{"features": [1.2, 2.3, 3.4, 4.5]}'

# Check status
curl http://localhost:5000/inference/{query_id}/status

Model Management

Adding New Models

1. Save your model in the models/saved_models/ directory:

   import joblib
   joblib.dump(your_model, 'models/saved_models/your_model.pkl')

2. Restart the service to load the new model:

   docker-compose restart web celery_worker

Supported Model Formats

• Scikit-learn models (.pkl, .joblib)
• Custom models with predict() method

Monitoring

Flower Dashboard

Access the Celery monitoring dashboard at: http://localhost:5555

Health Checks

• API health: http://localhost:5000/health
• Service status included in health response

Configuration

Environment variables can be set in env.example:

# Flask Configuration
SECRET_KEY=your-secret-key-here
FLASK_ENV=development

# MongoDB Configuration
MONGO_URI=mongodb://localhost:27017/inferno

# Celery Configuration
CELERY_BROKER_URL=redis://localhost:6379/0
CELERY_RESULT_BACKEND=redis://localhost:6379/0

Development

Project Structure

Inferno/
├── app.py                 # Main Flask application
├── models/
│   ├── __init__.py
│   └── ml_model.py       # ML model service
├── utils/
│   ├── __init__.py
│   └── validators.py     # Input validation
├── scripts/
│   ├── init_models.py    # Initialize sample models
│   └── test_api.py       # API testing script
├── requirements.txt      # Python dependencies
├── Dockerfile           # Container configuration
├── docker-compose.yml   # Multi-service setup
└── README.md           # This file

Running Tests

python scripts/test_api.py

Production Deployment

1. Update environment variables for production
2. Use production-grade WSGI server (Gunicorn is already configured)
3. Set up proper logging and monitoring
4. Configure reverse proxy (nginx) if needed
5. Set up SSL/TLS for HTTPS

Troubleshooting

Common Issues

1. Connection refused: Ensure all services are running
2. Model not found: Check model files in models/saved_models/
3. Celery worker not processing: Check Redis connection
4. MongoDB connection failed: Verify MongoDB is running and accessible

Logs

# View all logs
docker-compose logs

# View specific service logs
docker-compose logs web
docker-compose logs celery_worker

License

This project is open source and available under the MIT License.