Skip to content

fix(ci): retry gates on tenant quota; per-ref concurrency dedupe#221

Merged
Jesssullivan merged 1 commit into
mainfrom
fix/gates-quota-retry
Jul 7, 2026
Merged

fix(ci): retry gates on tenant quota; per-ref concurrency dedupe#221
Jesssullivan merged 1 commit into
mainfrom
fix/gates-quota-retry

Conversation

@Jesssullivan

Copy link
Copy Markdown
Owner

#219's repo-wide concurrency group turned quota storms into CANCELLED required checks: GitHub keeps only the newest pending job per group and evicts the middle ones — observed on run 28849318814 (#217 gates) and main's post-#220 run (09c1dd0, gate cancelled → main unverified).

Replacement:

  • Per-ref group (gf-reapi-tenant-gates-${{ github.ref }}, cancel-in-progress) — a newer push supersedes its own stale run only; no cross-PR eviction.
  • Bounded exit-34 retry in scripts/bazel-cache-backed.sh (3 attempts, 75s·n + jitter backoff, GF_BAZEL_QUOTA_RETRIES override) — RESOURCE_EXHAUSTED only; every other exit code propagates untouched. Mirrors the tinyland.dev lane's proven 3-attempt pattern (fleet precedent, not drift).

Cell quota stays 4 per the 2026-07-07 ruling.

The shared concurrency group from #219 has GitHub eviction semantics:
only one pending job per group is kept, middle jobs are CANCELLED
(run 28849318814 + main 09c1dd0) — fatal now that bazel-remote-gates
is a required check. Replace with per-ref dedupe + a bounded exit-34
(RESOURCE_EXHAUSTED) retry in bazel-cache-backed.sh, mirroring the
tinyland.dev lane's 3-attempt pattern. Other exit codes propagate
immediately; GF_BAZEL_QUOTA_RETRIES overrides the bound.
@Jesssullivan
Jesssullivan enabled auto-merge (squash) July 7, 2026 07:56
@Jesssullivan
Jesssullivan merged commit e14f1ce into main Jul 7, 2026
5 of 6 checks passed
@Jesssullivan
Jesssullivan deleted the fix/gates-quota-retry branch July 14, 2026 09:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant