[ACTP] orchestrate par-control tasks - #54593
Conversation
Go Package Import DifferencesBaseline: 296059f
|
|
🎯 Code Coverage (details) 🔗 Commit SHA: 2a3b060 | Docs | View more details | Give us feedback! |
Files inventory check summaryFile checks results against ancestor 296059ff: Results for datadog-agent_7.84.0~devel.git.459.2a3b060.pipeline.132944303-1_amd64.deb:No change detected Results for datadog-iot-agent_7.84.0~devel.git.459.2a3b060.pipeline.132944303-1_amd64.deb:No change detected |
b3f1b76 to
7d85848
Compare
a8be707 to
082615f
Compare
Static quality checks❌ Please find below the results from static quality gates Error
Gate failure full details
Static quality gate failures prevent this PR from merging! Successful checksInfo
3 successful checks with minimal change (< 2 KiB)
|
7d85848 to
8f87177
Compare
3292a06 to
940c82e
Compare
8f87177 to
2efe3f7
Compare
Regression DetectorRegression Detector ResultsMetrics dashboard Baseline: c45db9e Optimization Goals: ✅ No significant changes detected
|
| perf | experiment | goal | Δ mean % | Δ mean % CI | trials | links |
|---|---|---|---|---|---|---|
| ➖ | quality_gate_metrics_logs | memory utilization | +1.50 | [+1.27, +1.74] | 1 | Logs bounds checks dashboard |
| ➖ | quality_gate_idle_all_features | memory utilization | +0.48 | [+0.44, +0.52] | 1 | Logs bounds checks dashboard |
| ➖ | quality_gate_security_no_fs_load | memory utilization | +0.34 | [+0.26, +0.43] | 1 | Logs bounds checks dashboard |
| ➖ | quality_gate_idle | memory utilization | +0.32 | [+0.28, +0.36] | 1 | Logs bounds checks dashboard |
| ➖ | dsd_uds_10mb_3k_timestamped_contexts_memory | memory utilization | +0.11 | [-0.12, +0.33] | 1 | Logs |
| ➖ | quality_gate_security_mean_fs_load | memory utilization | +0.04 | [+0.00, +0.07] | 1 | Logs bounds checks dashboard |
| ➖ | quality_gate_private_action_runner | memory utilization | -0.14 | [-0.25, -0.02] | 1 | Logs bounds checks dashboard |
| ➖ | quality_gate_security_idle | memory utilization | -0.14 | [-0.20, -0.09] | 1 | Logs bounds checks dashboard |
| ➖ | quality_gate_logs | % cpu utilization | -0.29 | [-1.16, +0.58] | 1 | Logs bounds checks dashboard |
| ➖ | dsd_uds_10mb_3k_timestamped_contexts_cpu | % cpu utilization | -4.51 | [-4.81, -4.21] | 1 | Logs |
Bounds Checks: ✅ Passed
| perf | experiment | bounds_check_name | replicates_passed | observed_value | links |
|---|---|---|---|---|---|
| ✅ | quality_gate_idle | intake_connections | 10/10 | 4 = 4 | bounds checks dashboard |
| ✅ | quality_gate_idle | memory_usage | 10/10 | 173.03MiB ≤ 178MiB | bounds checks dashboard |
| ✅ | quality_gate_idle | total_bytes_received | 10/10 | 747.73KiB ≤ 819.20KiB | bounds checks dashboard |
| ✅ | quality_gate_idle_all_features | intake_connections | 10/10 | 4 = 4 | bounds checks dashboard |
| ✅ | quality_gate_idle_all_features | memory_usage | 10/10 | 520.91MiB ≤ 538MiB | bounds checks dashboard |
| ✅ | quality_gate_idle_all_features | total_bytes_received | 10/10 | 1.15MiB ≤ 1.25MiB | bounds checks dashboard |
| ✅ | quality_gate_logs | intake_connections | 10/10 | 16 ≤ 40 | bounds checks dashboard |
| ✅ | quality_gate_logs | memory_usage | 10/10 | 213.11MiB ≤ 229MiB | bounds checks dashboard |
| ✅ | quality_gate_logs | missed_bytes | 10/10 | 0B = 0B | bounds checks dashboard |
| ✅ | quality_gate_logs | total_bytes_received | 10/10 | 263.60MiB ≤ 292MiB | bounds checks dashboard |
| ✅ | quality_gate_metrics_logs | cpu_usage | 10/10 | 400.48 ≤ 2000 | bounds checks dashboard |
| ✅ | quality_gate_metrics_logs | intake_connections | 10/10 | 19 ≤ 40 | bounds checks dashboard |
| ✅ | quality_gate_metrics_logs | memory_usage | 10/10 | 420.30MiB ≤ 453MiB | bounds checks dashboard |
| ✅ | quality_gate_metrics_logs | missed_bytes | 10/10 | 0B = 0B | bounds checks dashboard |
| ✅ | quality_gate_metrics_logs | total_bytes_received | 10/10 | 0.94GiB ≤ 1.04GiB | bounds checks dashboard |
| ✅ | quality_gate_private_action_runner | memory_usage | 10/10 | 71.93MiB ≤ 76MiB | bounds checks dashboard |
| ✅ | quality_gate_security_idle | cpu_usage | 10/10 | 29.27 ≤ 100 | bounds checks dashboard |
| ✅ | quality_gate_security_idle | memory_usage | 10/10 | 325.35MiB ≤ 335MiB | bounds checks dashboard |
| ✅ | quality_gate_security_mean_fs_load | cpu_usage | 10/10 | 69.97 ≤ 200 | bounds checks dashboard |
| ✅ | quality_gate_security_mean_fs_load | memory_usage | 10/10 | 306.22MiB ≤ 314MiB | bounds checks dashboard |
| ✅ | quality_gate_security_no_fs_load | cpu_usage | 10/10 | 21.73 ≤ 100 | bounds checks dashboard |
| ✅ | quality_gate_security_no_fs_load | memory_usage | 10/10 | 310.07MiB ≤ 343MiB | bounds checks dashboard |
Explanation
Confidence level: 90.00%
Effect size tolerance: |Δ mean %| ≥ 5.00%
Performance changes are noted in the perf column of each table:
- ✅ = significantly better comparison variant performance
- ❌ = significantly worse comparison variant performance
- ➖ = no significant change in performance
A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".
For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:
-
Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.
-
Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that if our statistical model is accurate, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.
-
Its configuration does not mark it "erratic".
CI Pass/Fail Decision
✅ Passed. All Quality Gates passed.
- quality_gate_security_idle, bounds check cpu_usage: 10/10 replicas passed. Gate passed.
- quality_gate_security_idle, bounds check memory_usage: 10/10 replicas passed. Gate passed.
- quality_gate_idle_all_features, bounds check intake_connections: 10/10 replicas passed. Gate passed.
- quality_gate_idle_all_features, bounds check memory_usage: 10/10 replicas passed. Gate passed.
- quality_gate_idle_all_features, bounds check total_bytes_received: 10/10 replicas passed. Gate passed.
- quality_gate_private_action_runner, bounds check memory_usage: 10/10 replicas passed. Gate passed.
- quality_gate_logs, bounds check missed_bytes: 10/10 replicas passed. Gate passed.
- quality_gate_logs, bounds check total_bytes_received: 10/10 replicas passed. Gate passed.
- quality_gate_logs, bounds check memory_usage: 10/10 replicas passed. Gate passed.
- quality_gate_logs, bounds check intake_connections: 10/10 replicas passed. Gate passed.
- quality_gate_security_no_fs_load, bounds check cpu_usage: 10/10 replicas passed. Gate passed.
- quality_gate_security_no_fs_load, bounds check memory_usage: 10/10 replicas passed. Gate passed.
- quality_gate_idle, bounds check intake_connections: 10/10 replicas passed. Gate passed.
- quality_gate_idle, bounds check memory_usage: 10/10 replicas passed. Gate passed.
- quality_gate_idle, bounds check total_bytes_received: 10/10 replicas passed. Gate passed.
- quality_gate_metrics_logs, bounds check cpu_usage: 10/10 replicas passed. Gate passed.
- quality_gate_metrics_logs, bounds check intake_connections: 10/10 replicas passed. Gate passed.
- quality_gate_metrics_logs, bounds check memory_usage: 10/10 replicas passed. Gate passed.
- quality_gate_metrics_logs, bounds check missed_bytes: 10/10 replicas passed. Gate passed.
- quality_gate_metrics_logs, bounds check total_bytes_received: 10/10 replicas passed. Gate passed.
- quality_gate_security_mean_fs_load, bounds check cpu_usage: 10/10 replicas passed. Gate passed.
- quality_gate_security_mean_fs_load, bounds check memory_usage: 10/10 replicas passed. Gate passed.
2efe3f7 to
f1a50c2
Compare
940c82e to
a984166
Compare
f1a50c2 to
43371d3
Compare
a984166 to
f85a738
Compare
43371d3 to
a8463ff
Compare
984a3fd to
eb67245
Compare
c3d80ef to
84005d4
Compare
349cba4 to
8aac9cd
Compare
84005d4 to
adec48f
Compare
a3d07d0 to
cb06bd2
Compare
86614f2 to
ac84aa8
Compare
cb06bd2 to
7a4848d
Compare
3f91e32 to
c68c4c6
Compare
7a4848d to
ceaaf92
Compare
c68c4c6 to
9cfc7d0
Compare
93b15fa to
61de2b7
Compare
9cfc7d0 to
1aa18da
Compare
61de2b7 to
353ed4b
Compare
665dbfb to
f4cfc0a
Compare
0b063cb to
ecfb76e
Compare
f4cfc0a to
49d6ce1
Compare
ecfb76e to
a1c618c
Compare
49d6ce1 to
3763e5e
Compare
a1c618c to
28bfa69
Compare
3763e5e to
1cdd762
Compare
28bfa69 to
31ba15d
Compare
1cdd762 to
c139a60
Compare
31ba15d to
530c22e
Compare
c139a60 to
42177f2
Compare
530c22e to
22d2f12
Compare
42177f2 to
4359fe9
Compare
22d2f12 to
3b43251
Compare
4359fe9 to
1c989c7
Compare
3b43251 to
e219c39
Compare
1c989c7 to
738ab1f
Compare
What does this PR do?
Adds the production
par-controlorchestration loop. It dequeues tasks with bounded concurrency, starts the executor on demand, synchronizes signing keys, dispatches actions, and publishes outcomes. Executor idle shutdown is self-owned by the Go executor rather than driven bypar-controlprocess-manager calls.Task heartbeats run from dequeue through cold start, execution, and terminal publication. Runner health checks are independent of task flow, and shutdown drains in-flight work without calling
dd-procmgrdfrom the shutdown path.Validation
dda env dev run -- bazel test //pkg/privateactionrunner/par-control:par-control_testdda env dev run -- env -u PKG_CONFIG_LIBDIR cargo clippy --manifest-path pkg/privateactionrunner/par-control/Cargo.toml --all-targets -- -D warningsStack 7 of 9; based on #54592 and followed by #54594.