feat: Sentinel Phases 1-4 — classifier service, optimizer pipeline, local infra, and observability

[VjayRam]

Summary

Completes Phases 1–4 of the Sentinel content safety platform, establishing the full local development stack from model optimization through live inference and observability.

• Phase 1 — FastAPI classifier service with ONNX Runtime inference, Prometheus metrics, dynamic request batching, async PostgreSQL persistence, and Dockerfile
• Phase 2 — ONNX optimization pipeline: FP32 export → O2 graph optimization → INT8 dynamic quantization; all three stages uploaded to MinIO with pipeline report
• Phase 3 — Local Kubernetes infrastructure via Terraform: PostgreSQL (model_registry + classifications), MongoDB (flagged_content), MinIO (models/ + datasets/ buckets); dev-start.sh script to spin everything up in one command
• Phase 4 — Prometheus (scrapes classifier at host.k3d.internal:8000) and Grafana (auto-provisioned datasource) deployed into the cluster

Key design decisions captured

• sync def route for /classify — ONNX Runtime session.run() is a blocking C call; async def would stall the event loop
• No /reload endpoint — model upgrades go through kubectl rollout restart; in-process reload with multiple replicas causes a silent version split
• model_registry PostgreSQL table as the source of truth — classifier queries it on startup, downloads the active model from MinIO, caches at /tmp/sentinel-model-cache/<run-id>/int8/
• ON CONFLICT DO NOTHING on all DB writes — idempotent inserts survive Kafka reprocessing and pod restarts
• MinIO fallback — if MinIO is unreachable at optimization time, the local absolute path is stored in model_registry.model_path; download.py handles both formats transparently
• --address=0.0.0.0 on all kubectl port-forward commands — required for WSL2 so the Windows browser can reach services via localhost

What's in each file

Area	Files
Classifier service	services/classifier/ — main.py, model.py, db.py, download.py, batcher.py, metrics.py, schemas.py, Dockerfile
Optimizer pipeline	pipelines/optimizer/ — pipeline.py, export.py, optimize.py, quantize.py, upload.py, registry.py
Infrastructure	infra/terraform/local/ — PostgreSQL, MongoDB, MinIO, Prometheus, Grafana
Observability config	infra/prometheus/, infra/grafana/
Dev tooling	scripts/dev-start.sh, docs/local-dev.md
Tests	tests/test_classifier_api.py, tests/conftest.py

Test plan

• ./scripts/dev-start.sh — all 5 port-forward checks print ready, classifier starts, banner shows correct URLs
• curl http://localhost:8000/health → {"status":"ok","model":"..."}
• curl -X POST http://localhost:8000/classify -d '{"text":"test"}' → label + score + latency
• http://localhost:9001 — MinIO console shows models/ and datasets/ buckets
• http://localhost:9090/targets — Prometheus classifier job shows State: UP
• http://localhost:3000 — Grafana loads with Prometheus datasource connected
• psql ... -c "SELECT * FROM model_registry;" — row present after optimizer run
• uv run pytest --tb=short -q — all tests pass without torch installed

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features

  • Added a local development stack for running the classifier, database, storage, and monitoring services together.
  • Introduced single and batch classification endpoints with model health checks and metrics.
  • Added model caching and staged artifact handling for smoother local and deployed runs.

• Bug Fixes

  • Reduced unnecessary dependency installs in CI to speed up checks and avoid large downloads.

• Documentation

  • Expanded setup guides for local development, observability, model lifecycle, and infrastructure.

• Tests

  • Added API coverage for classification, batching, health, and metrics endpoints.

[sourcery-ai]

Hi @VjayRam! 👋

Your private repo does not have access to Sourcery.

Please upgrade to continue using Sourcery ✨

[coderabbitai]

Warning

Review limit reached

@VjayRam, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 55 minutes and 38 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Free

Run ID: 032f8760-7a17-4541-bd67-d577ebedf237

📥 Commits

Reviewing files that changed from the base of the PR and between 9b7c3ad and 53dd9d0.

📒 Files selected for processing (5)

• pipelines/optimizer/pipeline.py
• services/classifier/batcher.py
• services/classifier/main.py
• services/classifier/model.py
• tests/test_classifier_api.py

📝 Walkthrough

Walkthrough

Adds local infrastructure for the classifier stack, a FastAPI classifier with model loading, batching, persistence, and metrics, an optimizer pipeline that uploads and registers artifacts, plus developer workflow scripts and documentation for running the stack locally.

Changes

Sentinel local stack and model lifecycle

Layer / File(s)	Summary
Local infrastructure and observability README.md, docker-compose.yml, infra/..., docs/local-dev.md	Adds the Compose stack, Terraform-managed Kubernetes services, Prometheus/Grafana configuration, and the related infra and endpoint documentation.
Classifier core runtime services/classifier/schemas.py, services/classifier/model.py, services/classifier/batcher.py, services/classifier/metrics.py, services/classifier/download.py, services/classifier/Dockerfile, services/classifier/pyproject.toml, services/classifier/explanation.md	Defines classifier schemas, ONNX model loading and inference, dynamic batching, Prometheus metrics, MinIO model downloads, the container image, and package docs.
Classifier API and persistence services/classifier/db.py, services/classifier/main.py, services/classifier/explanation.md, tests/*, pyproject.toml, docs/local-dev.md	Adds async PostgreSQL helpers, the FastAPI lifespan and routes, and API tests plus classifier and PostgreSQL usage docs.
Optimizer pipeline and artifact flow pipelines/optimizer/..., pyproject.toml, .gitignore, README.md, docs/local-dev.md	Adds MinIO uploads, model registration, optimizer packaging, workspace wiring, artifact ignores, and model-lifecycle docs for optimizer outputs.
Local development launcher and guide scripts/dev-start.sh, docs/local-dev.md	Adds the local startup script and the remaining local-dev guide sections for MongoDB, Kubernetes inspection, port forwarding, and upcoming phases.
Production gaps audit ISSUES.md	Adds the Production Gaps audit and status tracker.

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~120 minutes

Poem

🐰 I hopped through logs and found a trail,
With carrots, caches, and metrics hale.
The batcher binkied, the models spun,
MinIO glinted in the sun.
Hooray—this stack now hums along!

---

Note

🎁 Summarized by CodeRabbit Free

Your organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Pro by visiting https://app.coderabbit.ai/login.

_{Comment @coderabbitai help to get the list of available commands.}