Aegis: Automated Anomaly Detection in System Data

Aegis is a production-grade full-stack MLOps web application designed to ingest system monitoring telemetry (e.g., CPU load, memory utilization, network requests) and detect anomalous events automatically. The project leverages an unsupervised Isolation Forest machine learning model on the backend and exposes a premium, glassmorphic dark-mode dashboard on the frontend for interactive exploration and live telemetry simulation.

---

🏗️ System Architecture

graph TD
    Client[React Frontend - Vite] -->|1. CSV Upload & Params| Nginx[Nginx Reverse Proxy]
    Client -->|2. Polling Live Telemetry| Nginx
    Nginx -->|3. Proxy /api/ to Port 8000| FastAPI[FastAPI Backend]
    FastAPI -->|4. Clean & Convert Timestamps| Preprocess[Pandas Data Cleaning]
    Preprocess -->|5. Fit / Predict / Score| IF[Isolation Forest Engine]
    IF -->|6. Save Serialized Model| Joblib[models/latest_model.joblib]
    IF -->|7. Export annotated CSV| Reports[data/annotated_uuid.csv]
    FastAPI -->|8. Return Stats & Plot Data| Client

Loading

---

🛠️ Technology Stack

• Frontend: React 19 + Vite, Tailwind CSS v4, Framer Motion (animations), Recharts (charts), Lucide (icons).
• Backend: Python 3.10+, FastAPI (high-performance async API), Pandas & NumPy (data wrangling), Scikit-Learn (Isolation Forest), Joblib (model serialization).
• DevOps: AWS EC2 (Ubuntu 22.04 t3.micro), Nginx (reverse proxy & static server), systemd (process daemon), Certbot Let's Encrypt (SSL HTTPS).

---

💻 Local Quickstart

Prerequisites

• Python 3.10+ installed.
• Node.js v18+ and npm installed.

Step 1: Run the Backend

1. Navigate to the backend folder:

   cd backend

2. Create and activate a Python virtual environment:

   python -m venv venv
   # On Windows:
   venv\Scripts\activate
   # On macOS/Linux:
   source venv/bin/activate

3. Install dependencies:

   pip install -r requirements.txt

4. Start the FastAPI development server:

   python -m uvicorn main:app --host 127.0.0.1 --port 8000 --reload

   • Swagger API documentation will be available at: http://127.0.0.1:8000/docs

Step 2: Run the Frontend

1. Open a new terminal window and navigate to the frontend folder:

   cd frontend

2. Install dependencies:

   npm install --legacy-peer-deps

3. Launch the Vite development server:

   npm run dev

   • The application will load at: http://localhost:5173

---

☁️ AWS EC2 Deployment Guide (t3.micro / Ubuntu 22.04)

This guide walks you through deploying both the frontend and backend on the same AWS EC2 t3.micro instance using Nginx and systemd.

1. Configure AWS Security Groups

Make sure your EC2 instance's security group allows incoming traffic on the following ports:

• Port 22 (SSH): From your developer IP.
• Port 80 (HTTP): From anywhere (0.0.0.0/0).
• Port 443 (HTTPS): From anywhere (0.0.0.0/0).

2. Connect to your EC2 Instance

ssh -i "your-key.pem" ubuntu@your-ec2-public-dns

3. Clone and Run the Automation Script

1. Clone your project code or upload it using SCP/SFTP to /var/www/.
2. Give execution permissions to the deployment script:

   chmod +x deploy.sh

3. Run the automated deployment script with your registered domain name (or raw public IP):

   sudo ./deploy.sh anomaly.yourdomain.com

4. Enable SSL/HTTPS with Let's Encrypt Certbot

To secure the application with HTTPS, run Certbot (which was installed by deploy.sh):

sudo certbot --nginx -d anomaly.yourdomain.com

Follow the interactive prompts. Certbot will automatically configure the SSL certificates and modify your Nginx file to redirect HTTP to HTTPS.

---

⚡ Low-Memory (1GB RAM) t3.micro Optimizations

Standard t3.micro instances can run out of memory (RAM) during Vite React builds, causing the process to terminate. Aegis resolves this using two major optimizations:

1. Linux Swap Space (2GB): The deploy.sh script automatically provisions a 2GB virtual memory swapfile. This allows the system to offload inactive RAM pages to disk, ensuring compilation does not crash.
2. Node Build Memory Cap: The build script executes with --max-old-space-size=512 to strictly limit the Node process heap allocation to 512MB:

   export NODE_OPTIONS="--max-old-space-size=512"
   npm run build

3. Nginx Cache Optimization: Nginx is configured to serve static assets with client-side cache control headers (Cache-Control "public, no-transform"), bypassing reverse-proxy execution for media/scripts, which minimizes CPU spikes.

---

🧪 Verification Plan

Once deployed, verify the system using these checks:

1. Service Health:
   • API health: curl http://localhost/health (should return "status": "healthy").
2. System Log Auditing:
   • Inspect backend output: sudo journalctl -u anomaly-backend -n 100 -f
   • Check Nginx error logs: sudo tail -f /var/log/nginx/error.log
3. Frontend Dashboard Testing:
   • Load your domain in a browser.
   • Toggle Light/Dark mode.
   • Go to Upload Data and run anomaly detection on the pre-generated sample dataset or custom CSV.
   • Go to ML Visualizer to filter metrics, inspect tooltip values, and download the annotated CSV report.
   • Go to Dashboard and turn on the Live Telemetry Stream to watch real-time points.