Avoid machine validation result/heartbeat deadlocks

[martinraumann]

Summary

• Add a dedicated DB helper that locks a machine_validation row with FOR NO KEY UPDATE.
• Use that helper before result persistence writes run-item/attempt state so result persistence follows a parent-run -> run-item/attempt lock order.
• Use the same parent-run -> run-item/attempt order in stale-attempt reconciliation before it writes attempt/run-item state.
• Keep the generic find_by_id() read path unchanged.

Why

This fixes a machine-validation deadlock caused by inconsistent lock ordering between concurrent validation paths.

Before this change, result persistence could lock child rows first and the parent run later:

1. persist_validation_result() called record_result().
2. record_result() wrote machine_validation_run_items and machine_validation_attempts.
3. A successful result insert into machine_validation_results fired a trigger that updated the parent machine_validation row.

Effective order: run item / attempt -> parent run.

The heartbeat path already used the opposite order:

1. record_heartbeat() updated the parent machine_validation row.
2. It then updated machine_validation_run_items.

Effective order: parent run -> run item.

Those two paths can deadlock when a result post and heartbeat for the same validation run/item overlap. This PR makes result persistence acquire the parent machine_validation lock before record_result(), so it follows the same parent-run -> run-item/attempt order as heartbeat.

The stale-attempt reconciler also writes attempt/run-item state before completing the parent validation. This PR updates that path to acquire the parent run lock first as well, so the touched machine-validation paths share the same ordering:

• Result persistence: parent run -> run item / attempt -> result insert trigger
• Heartbeat: parent run -> run item
• Stale-attempt reconciliation: parent run -> attempt / run item -> completion update

That removes the observed result-vs-heartbeat lock inversion and avoids introducing a result-vs-stale-reconciler inversion.

Validation

• cargo fmt passes. Stable rustfmt emits existing repo warnings for nightly-only rustfmt options.
• cargo clippy -p carbide-api-db --lib -- -D warnings passes.
• cargo clippy -p carbide-api-core --no-default-features --lib -- -D warnings passes.
• git diff --check passes.
• Changed-file secret-pattern check passed; matches were limited to CancellationToken identifiers.
• cargo clippy -p carbide-api-core --all-targets -- -D warnings is blocked locally on Apple Silicon because tss-esapi-sys v0.6.0 does not support the aarch64, darwin target without generated bindings.
• cargo test -p carbide-api-core --no-default-features test_machine_validation_m1_persists_selected_test_and_idempotent_result is blocked by the same local tss-esapi-sys target limitation through test/dev dependencies.

Security Evidence

Area	Result	Notes
SQL injection	Pass	New SQL is static and binds id via sqlx .bind(id).
Secret exposure	Pass	Changed-file pattern check found no credentials; only CancellationToken matched token.
Dependency/license scan	Not run locally	No dependency changes; cargo deny is not installed in this local environment.
Container scan	Not applicable locally	No container or image changes.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 21aa1e58-0d96-4b90-b703-411dd61a80f6

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}