feat: ONNX model optimization pipeline (Phase 2)#2
Conversation
Captures the deployment-boundary-first layout (pipelines/ vs services/) so future phases refactor toward it rather than away from it. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…peline with modular stages
…ORT compatibility issues with O2-optimized custom ops.
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
|
Warning Review limit reached
More reviews will be available in 58 minutes and 54 seconds. Learn how PR review limits work. Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file). ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits. 🚦 How do rate limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Free Run ID: 📒 Files selected for processing (13)
Note 🎁 Summarized by CodeRabbit FreeYour organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Pro by visiting https://app.coderabbit.ai/login. Comment |
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Ran dev-start.sh end-to-end against a real k3d cluster and exercised it under real load for the first time this session — surfaced three bugs none of the unit tests, terraform validate, or standalone smoke tests could have caught: 1. services/classifier/model.py: reverted padding="max_length" (#8's earlier "fix"). Every batch was tokenizing to the full 512 tokens regardless of actual input length — reproduced live, this OOM-killed the classifier pod (1Gi limit) twice under realistic load. The "latency doesn't depend on batch content" benefit isn't worth crashing the service; verified the revert holds memory flat (~324Mi) through a 300-trace sustained load test with zero restarts. 2. services/classifier/main.py: fixed #43's registration-skip check. download_model() always caches a MinIO-backed model under a local directory (/tmp/sentinel-model-cache/...), so _classifier.model_path is *always* an absolute local path — checking model_path.startswith("/") to detect "non-portable local fallback" therefore skipped registration for the common MinIO-backed case too. Every classification write then violated classifications_model_version_fkey, since the model_version never existed in model_registry. Now uses `model_dir is not None` (only None for a genuine local-only fallback) and registers the original MinIO path from the registry lookup, not the local cache path. 3. services/stream-processor/main.py: _pg_write only caught OperationalError (connection lost) for its reconnect logic. A ForeignKeyViolation from bug #2 above is a different failure mode — the connection stays alive but its transaction is aborted, so every subsequent command fails with InFailedSqlTransaction until rolled back. Without a rollback, the long-lived connection stayed permanently broken after the first such error — reproduced live, this stalled all PostgreSQL writes indefinitely even after bug #2 was fixed and redeployed, until this rollback was added. Verified end-to-end after all three fixes: dev-start.sh completes with zero errors, Kafka survives a real pod restart with identical topic/ consumer-group state before and after, and a fresh simulate-traces.py run shows classifications/flagged_content counts increasing correctly with zero duplicates and zero consumer lag. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
1 participant
Summary
models/<run-id>/,logs/optimizer/<run-id>/)main_exportfor export, ORT'squantize_dynamicdirectly for quantization) after discovering breaking changes in the Optimum 2.x releaseChanges
pipelines/optimizer/export.pymain_exportfromoptimum.exporters.onnx— correct API for optimum 2.xpipelines/optimizer/optimize.pyORTOptimizer; fuses Attention, SkipLayerNormalization, GeLUpipelines/optimizer/quantize.pyquantize_dynamicwithDefaultTensorTypeextra option to handle Microsoft-domain custom ops introduced by O2pipelines/optimizer/pipeline.pyreport.jsonper runpipelines/optimizer/explanation.mdpipelines/evaluation/README.mdCLAUDE.md.gitignorelogs/Verified
VijayRam1812/content-classifier-robertareport.jsonwritten correctly with run ID, stage durations, and artifact pathsNotes
Two optimum 2.x compatibility issues discovered and fixed:
opsetparameter removed fromORTModelForSequenceClassification.from_pretrained— switched tomain_exportcom.microsoftdomain ops (Attention,SkipLayerNormalization) that break ORT's quantizer type inference — fixed by dropping toquantize_dynamicwithextra_options={"DefaultTensorType": onnx.TensorProto.FLOAT}sinceQuantizationConfigdoes not expose this escape hatch🤖 Generated with Claude Code