Skip to content

TODO.md

Latest commit

 

History

History
338 lines (272 loc) · 13.2 KB

TODO.md

File metadata and controls

338 lines (272 loc) · 13.2 KB

TODO - Stackdog Security

Active Sprint (Phase 1: Foundation & eBPF Collectors)

Week 1-2: Project Scaffolding

  • TASK-001: Create new project structure for security modules

    • Status: ✅ COMPLETE
    • Summary: Created complete module structure, 38 TDD tests, eBPF crate
    • Docs: docs/tasks/TASK-001.md, docs/tasks/TASK-001-SUMMARY.md

  • TASK-002: Define security event types

    • Status: ✅ COMPLETE (Code Complete)
    • Summary: Implemented SyscallEvent, SecurityEvent, validation, stream types
    • Tests: 64 tests created
    • Docs: docs/tasks/TASK-002.md, docs/tasks/TASK-002-SUMMARY.md
    • Note: Full test execution blocked by external dependency conflicts

  • TASK-003: Setup aya-rs eBPF integration

    • Status: ✅ COMPLETE
    • Summary: Implemented EbpfLoader, SyscallMonitor, Kernel compatibility, RingBuffer
    • Tests: 35+ tests created
    • Docs: docs/tasks/TASK-003.md, docs/tasks/TASK-003-SUMMARY.md
    • Note: Stub implementation ready, actual eBPF programs in TASK-004

  • TASK-004: Implement syscall event capture

    • Status: ✅ COMPLETE
    • Summary: Implemented enrichment, container detection, eBPF types, 40+ tests
    • Tests: 40+ tests created
    • Docs: docs/tasks/TASK-004.md, docs/tasks/TASK-004-SUMMARY.md
    • Note: Infrastructure ready, eBPF programs need kernel implementation

  • TASK-005: Create rule engine infrastructure

    • Status: ✅ COMPLETE
    • Summary: Implemented RuleEngine, Signature database, 5 built-in rules, 59+ tests
    • Tests: 59+ tests created
    • Docs: docs/tasks/TASK-005.md, docs/tasks/TASK-005-SUMMARY.md

  • TASK-006: Implement signature-based detection

    • Status: ✅ COMPLETE
    • Summary: Implemented pattern matching, threat scoring, detection stats, 41+ tests
    • Tests: 41+ tests created
    • Docs: docs/tasks/TASK-006.md, docs/tasks/TASK-006-SUMMARY.md

  • TASK-007: Implement alert system

    • Status: ✅ COMPLETE
    • Summary: Implemented alert management, deduplication, 4 notification channels, 52+ tests
    • Tests: 52+ tests created
    • Docs: docs/tasks/TASK-007.md, docs/tasks/TASK-007-SUMMARY.md

  • TASK-008: Implement firewall integration

    • Status: ✅ COMPLETE
    • Summary: Implemented nftables, iptables, quarantine, automated response, 44+ tests
    • Tests: 44+ tests created
    • Docs: docs/tasks/TASK-008.md, docs/tasks/TASK-008-SUMMARY.md

  • TASK-009: Implement web dashboard

    • Status: ✅ COMPLETE (Foundation)
    • Summary: Implemented React dashboard, TypeScript types, API/WebSocket services, 15+ tests
    • Tests: 15+ tests created
    • Docs: docs/tasks/TASK-009.md, docs/tasks/TASK-009-SUMMARY.md
    • Note: Core components complete, stubs for AlertPanel, ContainerList, ThreatMap

  • TASK-010: Complete dashboard components

    • Status: ✅ COMPLETE
    • Summary: Implemented full AlertPanel, ContainerList, ThreatMap with charts, 20+ tests
    • Tests: 20+ tests created
    • Docs: docs/tasks/TASK-010.md, docs/tasks/TASK-010-SUMMARY.md
    • Note: All dashboard components complete, backend API endpoints needed

  • TASK-011: Implement backend API endpoints

    • Status: ✅ COMPLETE
    • Summary: Implemented 10 REST API endpoints + WebSocket handler, 17 tests
    • Tests: 17 tests created
    • Docs: docs/tasks/TASK-011.md, docs/tasks/TASK-011-SUMMARY.md
    • Note: All API endpoints ready for dashboard integration

  • TASK-012: Integration testing and release

    • Create src/collectors/ directory structure
    • Create src/events/ module for event types
    • Create src/rules/ module for rule engine
    • Update Cargo.toml with new dependencies (aya, candle-core)
    • Create eBPF build pipeline
    • Files: src/collectors/mod.rs, src/events/mod.rs, src/rules/mod.rs
    • Tests: Test module structure and imports

  • TASK-002: Define security event types

    • Create SyscallEvent struct with fields (pid, uid, syscall_type, timestamp)
    • Create SecurityEvent enum (Syscall, Network, Container, Alert)
    • Implement From/Into traits for conversions
    • Add builder pattern for complex event types
    • Files: src/events/syscall.rs, src/events/security.rs
    • Tests: test_syscall_event_creation, test_security_event_conversion

  • TASK-003: Setup aya-rs eBPF integration

    • Add aya dependency to Cargo.toml
    • Create eBPF program skeleton
    • Implement eBPF program loader
    • Create syscall tracepoint program
    • Files: src/collectors/ebpf/mod.rs, src/collectors/ebpf/loader.rs, ebpf/src/syscalls.rs
    • Tests: test_ebpf_program_load, test_tracepoint_attach

  • TASK-004: Implement syscall event capture

    • Implement execve syscall monitoring
    • Implement connect syscall monitoring
    • Implement open syscall monitoring
    • Create event ring buffer for eBPF events
    • Files: src/collectors/ebpf/syscalls.rs
    • Tests: test_execve_capture, test_connect_capture, test_open_capture

Week 3-4: Rule Engine

  • TASK-005: Create rule engine infrastructure

    • Define Rule trait with evaluate() method
    • Create RuleEngine struct for rule management
    • Implement rule registration system
    • Add rule priority and ordering
    • Files: src/rules/engine.rs, src/rules/rule.rs
    • Tests: test_rule_registration, test_rule_evaluation, test_rule_priority

  • TASK-006: Implement signature-based detection

    • Create Signature struct for threat patterns
    • Implement pattern matching for syscalls
    • Add known threat signatures (crypto miner, scanner, etc.)
    • Create signature database
    • Files: src/rules/signatures.rs, src/rules/database.rs
    • Tests: test_signature_match, test_known_threat_detection

  • TASK-007: Docker events integration

    • Add bollard event stream listener
    • Implement container start/stop events
    • Implement container lifecycle monitoring
    • Add Docker event to SecurityEvent conversion
    • Files: src/collectors/docker_events.rs
    • Tests: test_docker_event_stream, test_container_lifecycle_tracking

Phase 2: Firewall & Response Engine (Weeks 5-6)

Week 5: nftables Integration

  • TASK-008: Implement nftables backend

    • Add netlink-packet-route dependency
    • Create NfTables wrapper struct
    • Implement rule addition/removal
    • Implement batch updates for performance
    • Files: src/firewall/nftables.rs
    • Tests: test_nft_rule_add, test_nft_rule_remove, test_nft_batch_update

  • TASK-009: Implement iptables fallback

    • Create Iptables wrapper struct
    • Implement rule management
    • Add feature flag for backend selection
    • Files: src/firewall/iptables.rs
    • Tests: test_ipt_rule_add, test_ipt_rule_remove

Week 6: Response Actions

  • TASK-010: Container quarantine system

    • Implement network isolation for containers
    • Implement process blocking
    • Add quarantine state tracking
    • Create quarantine rollback mechanism
    • Files: src/firewall/quarantine.rs
    • Tests: test_container_quarantine, test_quarantine_rollback

  • TASK-011: Response action pipeline

    • Define Action trait with execute() method
    • Create action chain for complex responses
    • Implement action logging and audit
    • Add action retry logic
    • Files: src/response/actions.rs, src/response/pipeline.rs
    • Tests: test_action_execution, test_action_chain, test_action_retry

Phase 3: ML Anomaly Detection (Weeks 7-10)

Week 7-8: Candle Integration

  • TASK-012: Setup Candle ML backend

    • Add candle-core and candle-nn dependencies
    • Create CandleBackend struct
    • Implement model loading
    • Create tensor conversion utilities
    • Files: src/ml/candle_backend.rs
    • Tests: test_candle_initialization, test_model_loading, test_tensor_conversion

  • TASK-013: Feature extraction pipeline

    • Define SecurityFeatures struct
    • Implement feature normalization
    • Create feature vector from events
    • Add feature statistics tracking
    • Files: src/ml/features.rs
    • Tests: test_feature_extraction, test_feature_normalization, test_feature_vector_creation

Week 9-10: Anomaly Detection

  • TASK-014: Isolation Forest implementation

    • Implement Isolation Forest model with Candle
    • Create training pipeline
    • Implement anomaly scoring
    • Add model persistence
    • Files: src/ml/anomaly.rs, src/ml/models/isolation_forest.rs
    • Tests: test_isolation_forest_training, test_anomaly_scoring, test_model_persistence

  • TASK-015: Baseline learning system

    • Implement baseline statistics collection
    • Create adaptive baseline updates
    • Add baseline persistence to database
    • Implement baseline drift detection
    • Files: src/baselines/learning.rs, src/database/baselines.rs
    • Tests: test_baseline_collection, test_baseline_update, test_drift_detection

  • TASK-016: Threat scoring system

    • Define ThreatScore enum (Normal, Low, Medium, High, Critical)
    • Implement score calculation from ML output
    • Add score aggregation for multiple events
    • Create score threshold configuration
    • Files: src/ml/scorer.rs
    • Tests: test_threat_score_calculation, test_score_aggregation, test_threshold_detection

Phase 4: Event Correlation & Alerting (Weeks 11-12)

  • TASK-017: Event correlation engine

    • Implement temporal correlation (time-window based)
    • Implement pattern correlation (multi-event patterns)
    • Create correlation rules
    • Files: src/correlator/engine.rs
    • Tests: test_temporal_correlation, test_pattern_correlation

  • TASK-018: Alert rules engine

    • Define alert rule DSL
    • Implement rule evaluation
    • Add alert severity levels
    • Files: src/alerting/rules.rs
    • Tests: test_alert_rule_evaluation, test_severity_assignment

  • TASK-019: Notification system

    • Implement Slack notifications
    • Implement email notifications
    • Implement webhook notifications
    • Add notification deduplication
    • Files: src/alerting/notifications.rs, src/alerting/dedup.rs
    • Tests: test_slack_notification, test_email_notification, test_deduplication

Phase 5: Web Dashboard (Weeks 13-16)

  • TASK-020: Security dashboard API

    • Create dashboard endpoints
    • Implement WebSocket for real-time updates
    • Add threat visualization data
    • Files: src/api/security.rs, src/api/websocket.rs
    • Tests: test_dashboard_api, test_websocket_stream

  • TASK-021: React dashboard components

    • Create main Dashboard component
    • Create ThreatMap visualization
    • Create AlertPanel component
    • Create ContainerList with security status
    • Files: web/src/components/Dashboard.tsx, web/src/components/ThreatMap.tsx
    • Tests: Component unit tests

Phase 6: Hardening (Weeks 17-18)

  • TASK-022: Performance benchmarking

    • Add criterion benchmarks
    • Profile memory usage
    • Optimize hot paths
    • Files: benches/throughput.rs, benches/latency.rs

  • TASK-023: Security audit

    • Run cargo audit
    • Run cargo deny
    • Memory safety review
    • Dependency review

  • TASK-024: Integration tests

    • End-to-end threat detection test
    • Auto-quarantine workflow test
    • ML pipeline test
    • Files: tests/integration/full_stack_test.rs

Backlog (Future Phases)

  • BACKLOG-001: Auditd integration for additional event sources
  • BACKLOG-002: Network packet capture (optional, heavy)
  • BACKLOG-003: Graph neural networks for attack chain detection
  • BACKLOG-004: Autoencoder for unsupervised anomaly detection
  • BACKLOG-005: Kubernetes integration
  • BACKLOG-006: Multi-node cluster support
  • BACKLOG-007: Security compliance reporting (CIS, PCI-DSS)
  • BACKLOG-008: Threat intelligence integration (STIX/TAXII)

Technical Debt

  • DEBT-001: Remove legacy container management code (v1.0.0)
  • DEBT-002: Replaced Diesel with rusqlite + r2d2
  • DEBT-003: Migrate from actix-web 3.x to 4.x
  • DEBT-004: Improve error handling with thiserror
  • DEBT-005: Add comprehensive documentation comments

Notes

TDD Reminders

  • Write test first, then implementation
  • Run tests frequently (cargo watch -x test)
  • Keep tests fast and isolated
  • Use descriptive test names: test_<scenario>_<expected_result>

Code Quality

  • Run cargo fmt --all before commits
  • Run cargo clippy --all before commits
  • Keep functions under 50 lines
  • Follow DRY principle
  • Use builder pattern for complex objects

eBPF Development

  • eBPF programs require kernel 4.19+
  • Test on VM before production
  • Use bpftool for debugging
  • Keep eBPF programs simple (complexity limit)