CI pipeline speed: measured baseline + prioritized measures (target ~254s → ~150–190s)

[CybotTM]

Status — updated 2026-07-15

Measured on main (uncontended, CI workflow wall-clock): ~254s → ~166s (−33%).
Baseline c2f4b82a = 253s (post-QW1/QW2, pre-S1) → current b07dbcb7 = 166s (S1 + quick-wins on the flattened graph). The S1 PR run measured 169s independently. Under contention the absolute saving is larger (the old graph reached 280–363s; runs are now 166–253s).

Measure	State	PR	Measured / note
QW1 concurrency groups + cancel	✅ done	#625	targets contention bursts (median 455→254s projected)
QW2 E2E matrix 10 → 4 shards	✅ done	#625	−~500s runner-time/run, no wall-clock regression
QW3 LDAP networkTimeout	✅ done	#629	config-readable, 5s fail-fast default; bounds prod login hangs
QW4 bcrypt cost-4 test hashing	✅ done	#629	test env only, keeps $2y$ format
QW9 e2e video: on-first-retry	✅ done	#629	—
QW10 skip profiling build on PRs	✅ done	#629	—
S1 flatten job graph (drop setup)	✅ done	#630	253s → 166s main / 169s PR — 89s setup barrier + 872 MB artifact handoff removed
S4 Dockerfile stage reorder	✅ done	#631	devtools stage caches dev/e2e tooling below the code copy — verified: after a src change the whole stage stays CACHED. 0 PR-wall-clock; Docker Publish ~226→120s
QW7 vitest isolate: false	❌ dismissed	—	leaks the vi.mock/ESM module registry: 68 failed under --no-file-parallelism vs 576 passed reverted (forks pool masked it); gain ~0.3s (noise)
QW8 deps artifact tar.zst	❌ dismissed	—	subsumed by S1 (#630) — the deps artifact is gone entirely
QW5 zstd :e2e image	⬜ open	—	[EST] — needs a verification push
QW6 one bake for Docker Publish	⬜ open	—	0 wall-clock, Docker Publish −~60s
QW11 incremental tsc cache	⬜ open	—	−7s runner-time
S2 persist PHP tool caches	✅ done	#632	measured Lint 145→72s warm (−50%, cache-hit verified); Integration ~146s stays the gate — runner-time win, not wall-clock
S3 chromium-free :dev image	⬜ open	—	[EST]
S5 coverage xdebug → pcov	❌ dismissed	—	wrong trade: pcov is line-only, would foreclose branch/path coverage for ~18s. Kept xdebug; did S9 + enabled branch coverage instead
S6 paratest -p4	⬜ open	—	[EST]
S7 native Unit job	⬜ open	—	−~65s runner-time
S8 skip heavy jobs on docs-only	⬜ open	—	—
S9 coverage off PR path → nightly	✅ done	#633	coverage (unit/integration/e2e) now nightly + workflow_dispatch only; PR path coverage-free. Branch coverage enabled (Cobertura). NB xdebug branch coverage is ~6× slower → nightly job timeouts raised (#634)

Legend: ✅ done · 🔵 open PR · ⬜ open · ❌ dismissed

---

Summary

CI wall-clock on a PR is bimodal: ~254s for an isolated push, ~560s when 3–4 pushes land together. The ~200s gap is runner queueing, not runtime — 14–16 jobs per run compete for the (inferred) ~20-job public-repo runner cap. The intrinsic pipeline floor is ~254s.

Measured over CI runs 1330–1358 (2026-07-12..14) from the Actions API; per-run job+step timings retained. Estimates are marked [EST]; a full "not verified" list is at the bottom.

Baseline — uncontended critical path (~254s)

push → queue 3s
     → Setup Dependencies 78s   (37s of it = pulling the 872MB :e2e image)
     → wave 2 (slowest gates):
          Integration Tests 166s   ← gate in 5/10 runs
          Lint & Static     148.5s ← 4–23s behind
          slowest E2E shard up to 147s ← gate in the other 5/10
          Unit 107.5s, Frontend 74s (never gate)
     → CI Success 3s

Runner-time budget ≈ 1627s/run (~27 min); the 10-shard E2E matrix is 65% of it. Every one of 14 jobs pulls the same 872MB image (~8.6 min of pure pull/run per run). Structural facts (verified in ci.yml): no concurrency:, no paths: filters, no actions/cache in any of the 7 workflows.

Quick wins (S effort)

#	Measure	Wall-clock	Runner-time	Risk
✅ QW1	concurrency groups + cancel-in-progress (4 workflows)	455 → ~254s median (−44%)	shrinks bursts	full gain needs main-scope cancel or a merge queue (coarser bisect); PR-scope is risk-free
✅ QW2	E2E matrix 10 → 4 shards	0 regression (131s/shard < 166s gate)	−~500s/run	uneven Playwright sharding; N=4 keeps ~35s headroom
✅ QW3	LDAP networkTimeout (one test hangs 10.7s)	Integration −11s	—	low; also bounds prod login hangs
✅ QW4	Cheap test password hashing (bcrypt cost 4, not plaintext)	with QW3 caps at −17.5s (Lint takes over)	Unit −10s	keep cost 4 so hash-format asserts pass
⬜ QW5	Push :e2e image zstd-compressed [EST]	path −~40s if extraction-bound holds	−~4 min/run	verify one :e2e-zstd push first
⬜ QW6	One bake for all 3 Docker Publish targets	0 (never gates)	Docker Publish −~60s	coarser logs
❌ QW7	Vitest isolate: false (564/564 pass locally) — DISMISSED: leaks vi.mock/ESM registry, breaks under --no-file-parallelism	0	Frontend −14s	state leakage; 1-line revert
❌ QW8	Deps artifact as tar.zst (keeps .bin symlinks)	0	−9s/shard	subsumed by S1 — interim only
✅ QW9	E2E video: on-first-retry	0	−20–30s/run [EST]	loses first-attempt video on pass-on-retry
✅ QW10	Skip profiling image on pull_request	0	−17s/PR	profiling breakage surfaces on main push
⬜ QW11	Incremental tsc + cached .tsbuildinfo	0	−7s/run	self-healing

Structural (M effort)

• ✅ S1 — flatten the job graph (drop needs: setup, wave-2 self-installs). [ci: flatten the job graph — drop setup, per-job self-install (#624 S1) #630] The biggest uncontended-wall lever: composer install is 13s in-container vs the 78s Setup + handoff paid today → uncontended 258 → ~188s (−70s); measured 253 → 166s on main. Pair with QW2 (offsets the +100s runner-time from 10 shards self-installing). Subsumes QW8.
• ✅ S2 — persist PHP tool caches (actions/cache: PHPStan/PHPat/Rector/CS-Fixer). [ci: persist PHP static-analysis tool caches across runs (#624 S2) #632] Measured: Lint 145 → 72s warm (−50%), cache-hit verified (7 MB restored). PHPStan tmpDir + Rector cache moved to host-mounted var/cache; the deferred bit was they wrote to the ephemeral container /tmp. Runner-time win — Integration (~146s) stays the gate, so ~0 wall-clock until S5 cuts it.
• ⬜ S3 — chromium-free :dev image for the 4 non-browser jobs. Stacked with QW5 → ~10–12s pulls [EST].
• ✅ S4 — reorder Dockerfile stages (dev tools + chromium before COPY . .). [perf(ci): cache dev/e2e Docker tooling below the code copy (#624 S4) #631, CI green] Every commit re-runs a measured 68.3s of apt/pecl/chromium/composer because the dev+e2e stages build after the app-code copy → E2E build 131 → ~30s, Docker Publish 226 → ~120s. 0 CI wall-clock but faster image availability. Verified: a src change leaves the new devtools stage CACHED.
• ❌ S5 — CI coverage xdebug → pcov. DISMISSED. pcov is line-coverage-only and would foreclose branch/path coverage for a marginal ~18s. Kept xdebug; see S9 instead. Separately enabled branch coverage (branchCoverage=true → Cobertura) — the thing pcov could not do.
• ⬜ S6 — paratest -p4 with per-process DBs. Runner-time only until S2/QW2; DB-isolation is the real work [EST].
• ⬜ S7 — run the Unit job natively (setup-php, no Docker). −~65s/run; overlaps S1/S5.
• ⬜ S8 — skip heavy jobs on docs-only pushes (in-workflow paths-filter). ~455 → ~40s per docs-only push (real here: the two docs(adr-024) commits ran the full matrix). Must be in-workflow so the required CI Success check still resolves.
• ✅ S9 — coverage off the PR path → nightly. [ci: coverage off the PR path → nightly + branch coverage (#624 S9, drops S5) #633] unit/integration/e2e run coverage-free on PR & push; a nightly schedule + workflow_dispatch re-collect it (with branch coverage, Cobertura) and upload to Codecov. Only CI Success is required, so the per-PR Codecov uploads were safe to drop.

Projected "after" (uncontended wall-clock)

• Quick wins only: median ~235s verified (QW1 + QW3 + QW4); ~205s if the QW5 zstd estimate holds. Runner-time 1627 → 1050–1100s (−35%, dominated by QW2).
• Everything: ~150–190s, anchored at ~188s from S1 alone (measured arithmetic); the ~150s lower bound stacks several [EST] figures and is directional until a trial run. Runner-time → 850–900s (−45%). Docker Publish → ~110–120s.
• Actual so far: 166s on main with QW1/QW2 + step-2 quick wins + S1 landed — already inside the target band.

Recommended order

1. ✅ QW1 + QW2 — one tiny PR, most of the win (median −44%, runner-time −500s), lowest risk. [ci: concurrency groups + E2E 4 shards (measured speed wins from #624) #625]
2. ✅ QW3, QW4, QW9, QW10 — mechanical, independently mergeable. [ci: #624 step-2 quick wins (LDAP timeout, cheap test hashing, e2e video, PR profiling skip) #629] (QW7 dropped.)
3. ✅ S1 — the uncontended-wall step. [ci: flatten the job graph — drop setup, per-job self-install (#624 S1) #630]
4. ✅ S4 — Docker Publish + image-availability. [perf(ci): cache dev/e2e Docker tooling below the code copy (#624 S4) #631]
5. ✅/⬜ S2 [ci: persist PHP static-analysis tool caches across runs (#624 S2) #632, measured] + QW5 + S3 — the caching/image-diet block. S2 done (Lint −50% warm) but wall-clock-neutral until S5 (pcov) cuts the Integration gate — that is the next real wall-clock lever. QW5/S3 are runner-time [EST].

Not verified (estimates)

zstd pull speedup (QW5/S3); 412MB layer = chromium (S3); PHPStan cache hit rate (S2); pcov overhead (S5); paratest DB isolation (S6); the post-change composition band (computed from per-step medians, not a trial run); docs-only push frequency (S8); the ~20-job runner cap (inferred, not read from billing); xdebug/video overheads (S9/QW9). Full detail with per-measure arithmetic is in the analysis report.

[CybotTM]

Progress update — 2026-07-15

Working through the recommended order.

Shipped

Step 1 — QW1 + QW2 — merged in #625: concurrency groups + cancel-in-progress, and the E2E matrix cut 10 → 4 shards.

Step 2 — QW3, QW4, QW9, QW10 — merged in #629:

• QW3 LDAP networkTimeout (Laminas), read from service config with a 5s fail-fast default — also bounds prod login hangs.
• QW4 bcrypt cost-4 password hashing in the test env (was the inherited auto = Argon2id).
• QW9 Playwright video: retain-on-failure → on-first-retry (matches trace; CI runs with retries=2).
• QW10 skip the profiling image build on PRs.

Step 3 — S1 (flatten the job graph) — merged in #630. Removed the setup job and its ~872 MB deps artifact; each job self-installs (composer install; e2e also builds the SPA before the stack starts). Measured wall-clock 169s on the PR run, vs ~241–258s uncontended for the setup-based graph (the ~89s setup barrier + per-job artifact download removed). First CI attempt green; all 4 E2E shards pass with their self-built SPA.

Step 4 — S4 (Dockerfile stage reorder) — open in #631 (CI green, awaiting review). New devtools stage FROM base caches the code-independent dev/e2e tooling (git/curl, Xdebug, Symfony CLI, Node, Bun, the Playwright npm deps, Chromium) below the code copy. Verified locally: docker build --check clean on all 5 targets; docker buildx bake app-e2e builds; after a src change the whole devtools stage stays CACHED (only dev's COPY --from=deps re-runs). Xdebug/Chromium stay out of production/profiling/tools. Scope: 0 PR wall-clock (the docker job doesn't gate CI Success) — the gain is faster Docker Publish (~226→120s) and cache-miss builds.

Dropped

QW7 (vitest isolate:false) — unsafe. It leaks the vi.mock / ESM module registry, which no afterEach resets — only per-file isolation does; the default forks pool masks it. Reproduced with a controlled before/after: 68 failed under --no-file-parallelism vs 576 passed after reverting (identical ordering, only the flag flipped). Measured gain ~0.3s (noise). Excluded.

Remaining

• S2 — persist PHPStan/PHPat/Rector/CS-Fixer caches (actions/cache). Now the next gate-remover: after QW3/QW4, Lint is the gate that S2 cuts (148.5 → ~87s cold→warm). Needs a real cache-HIT verification on first push.
• S5 — CI coverage xdebug → pcov (Unit 33→18s, Integration 79→61s — cuts the current Integration gate). Needs a Dockerfile pcov add + a verification push. [EST].
• S3 / QW5 — chromium-free :dev image / zstd image push (runner-time). [EST].
• S8 — skip heavy jobs on docs-only pushes (in-workflow paths-filter so CI Success still resolves).