Make pinned CI tool downloads resilient to transient upstream failures

[doublewhy]

Plain-language summary

• Context: OpenRAE downloads checksum-pinned copies of Conftest, Vale, Gitleaks, and OSV Scanner before several required verification lanes.
• Problem: A single temporary GitHub Releases disconnect or HTTP 503 fails the entire lane before project code is tested. Multiple otherwise-clean PRs have required manual reruns for the same upstream event.
• Fix: Retry only transient transport and server failures with a small bounded backoff, while preserving the approved HTTPS origin, version, size limit, checksum, signature, and fail-closed behavior.

Evidence

On 2026-08-12, required checks for multiple independent PRs failed while acquiring Conftest, Vale, Gitleaks, or OSV Scanner. The observed failures were remote disconnects and HTTP 503 responses from GitHub Releases. The same commits passed once the upstream download succeeded, and the failures occurred before the affected verification stage could inspect project code.

Acceptance criteria

• Share one deterministic bounded retry policy across the pinned tool downloaders where practical.
• Cover Conftest, Vale, Gitleaks, and OSV Scanner acquisition, including checksum metadata where the owning installer uses it.
• Retry remote disconnects, timeouts, incomplete responses, connection failures, HTTP 408/429, and HTTP 5xx only.
• Do not retry checksum, signature, archive/type, size, identity, origin, version, or other policy failures.
• Honor Retry-After only within the total retry/time bound.
• Keep exact approved origins and pinned versions/digests; do not fall back to an unreviewed mirror.
• Preserve stable final diagnostics without leaking credentials, exception text, or response bodies.
• Add deterministic tests for success after transient failure, Retry-After, exhaustion, non-retryable HTTP failures, corrupt payloads, and the total attempt bound.
• Keep required CI lanes fail closed when all attempts fail.

Scope

This is CI/tool-acquisition reliability only. It does not make vulnerability findings advisory, relax proof isolation, or change the verified artifacts.

[doublewhy]

Additional CI evidence from the current successor PR set:

• PR fix(proof): make offline Isabelle replay portable on Ubuntu #1131, run https://github.com/OpenRAE/rae/actions/runs/31638606534: the unit, integration, contract, static, and proof work completed, while Vale and OSV acquisition failed with RemoteDisconnected.
• PR fix(runtime): reconcile workflow timeouts fail closed #1132, run https://github.com/OpenRAE/rae/actions/runs/31639240594: Conftest priming and OSV acquisition both failed with RemoteDisconnected before the affected checks could exercise project code.
• PR fix(release): gate publication on the verified release SHA #1138, run https://github.com/OpenRAE/rae/actions/runs/31638450185: the reusable canonical verifier passed unit, integration, contracts, static analysis, and the Isabelle proof, then failed only when the pinned Vale asset download disconnected.

These are independent branches and runner attempts with the same failure class. This reinforces the acceptance criterion: retry only bounded transport failures while preserving exact origin, version, digest/signature checks, and fail-closed behavior for any integrity mismatch.

[doublewhy]

Temporarily held pending post-#1113 evidence and the breaking-change freeze. Merged #1113 already provides one shared bounded retry helper for Conftest, Vale, Gitleaks, and OSV. Before resuming this issue, record at least one reproducible transient acquisition failure on a commit containing #1113 and narrow any follow-up to the missing behavior. PR #1140 remains draft and unmerged.

[Brad-Edwards]

Deferred. Points to larger issue of maturing artifact/package management in the CI/CD. Milestone created to mature that.

Custom HTTP stack will not be merged as it will be superseded by a mature package management solution.

[Brad-Edwards]

Deferred under #1168 and the Developer Package & Artifact Management milestone. #1168 blocks this issue through GitHub’s native dependency graph. The repository-owned HTTP transport direction is not an accepted implementation; #1137 will be dispositioned or re-scoped after the package/artifact-management architecture is accepted.