Drift-Aware Self-Evolving Intrusion Detection Framework for Non-Stationary Cloud Environments
Network intrusion detection in cloud computing faces the fundamental challenge of concept drift, where shifting traffic patterns, user behaviors, and new attack methodologies rapidly degrade static model performance. EvoIDS introduces a unified framework combining adaptive learning with open-set recognition to maintain robust operational effectiveness under temporally shifting, realistic cloud traffic conditions.
EvoIDS represents a paradigm shift from static classifiers to dynamic, self-healing intrusion detection systems.
- 🕰️ Realistic Temporal Evaluation: Highlights the pitfalls of random train-test splitting (which overstates performance) and demonstrates the severity of model degradation (up to 64% drop in operational performance) under realistic chronological deployments on the CICIDS2017 dataset.
- 🧬 Drift-Guided Self-Evolution: Introduces a Multi-Signal Health Monitor (MSHM) using Population Stability Index (PSI) and prediction confidence to trigger adaptive retraining, successfully recovering up to 99.5% accuracy.
- 🔍 Multi-Detector Open-Set Recognition (MDOM): Fuses autoencoder reconstruction, k-NN latent-space distance, Isolation Forest, and Mahalanobis distances using CDF-rank composition to flag 99.96% of PortScan, 91.53% of DDoS, and 85.74% of Bot zero-day attacks.
EvoIDS is comprised of three core coordinated subsystems designed to keep the intrusion detection robust over time.
| At the heart of EvoIDS is a high-efficiency LightGBM gradient boosting classifier that distinguishes malicious network activity from legitimate traffic. Unlike static classifiers, this subsystem is fully designed to have its parameters incrementally updated or fine-tuned seamlessly. |
Monitors incoming traffic for covariate and concept drift without relying on a single metric heuristic. It integrates:
|
A parallel processing stream identifying novel attacks by verifying flow conformity against known distributions. Uses 5 complementary detectors fused via Cumulative Distribution Function (CDF) rank aggregation to score the "unknownness" of each sample.
When evaluated chronologically, conventional static models suffer a catastrophic drop in performance. By leveraging a Replay Memory and triggering retraining only when drift thresholds are crossed, EvoIDS effectively adapts its decision boundaries.
Static models exhibit a sharp F1-score and Accuracy collapse when exposed to temporally subsequent data (Concept Drift).
EvoIDS seamlessly adapts and recovers predictive performance, showcasing an average gain of 24.7 percentage points compared to static deployments.
EvoIDS successfully maps and flags unknown inputs using Mahalanobis distance and latent-space analysis.
Mahalanobis distance separating known vs unknown classes.
Global Feature Importance determining attack attributes.
Per-Attack Detection Rate Comparison across different threat vectors.
Released under the Standard License. For full details, please refer to the research paper included in this repository.