asan + ubsan (full suite) failed on PR #759 with two checks that have nothing to do with that PR:
FAIL: HS27 aggregate body budget (got 000 (expected 503))
FAIL: HS28 server survives over-budget upload (got '')
Not deterministic: the same commit passed that job on its previous run, and locally make asan-http && bash tests/test_http_server.sh is 37/37. Both symptoms are empty responses (000, ''), i.e. the second server was never reachable — not a budget-logic failure. The harness misdiagnoses its own setup failure as a DoS-gate regression, which is the shape that wastes the most time when it fires in CI.
Three defects in the HS27/HS28 block (tests/test_http_server.sh:605-646), each measured on the dev box:
1. The listen port is drawn from inside the kernel's ephemeral range.
PORT2=$(( (RANDOM % 10000) + 50000 )) # 50000-59999
$ cat /proc/sys/net/ipv4/ip_local_port_range
32768 60999
The whole 50000-59999 window sits inside the ephemeral range, and this suite makes dozens of curl connections that each take an ephemeral port from it. The test can therefore lose the bind to its own client traffic. There is no collision check and no retry.
2. The readiness loop's real budget is ~3.6s, not the ~33s it looks like.
for _ in $(seq 1 30); do
curl -s --max-time 1 "http://127.0.0.1:$PORT2/ping" > /dev/null 2>&1 && break
sleep 0.1
done--max-time 1 only bounds the case where something is listening; a refused connection returns instantly, so this is effectively 30 × sleep 0.1:
readiness loop against a DEAD port took: 3.580882504s
For calibration, an asan-http server on this idle box is ready in 0.116s — a 31× margin, so slow start is not the likely trigger here, but the budget is far tighter than it reads.
3. Neither failure is detected. If all 30 probes fail the loop just ends and the test proceeds against a server that isn't there. A bind failure in the server process goes to /tmp/eigs_http_srv2_$$.log, which is never inspected and never uploaded as a CI artifact — so the log that would name the real cause is discarded at the moment it becomes interesting.
Suggested fix
- Pick the port outside the ephemeral range, or bind-probe and retry on collision.
- Make the readiness loop assert: if no probe succeeds,
fail with "server never came up" and dump /tmp/eigs_http_srv2_$$.log — a setup failure should say so instead of blaming the budget.
- Give the loop a wall-clock deadline rather than an iteration count, so its budget doesn't depend on whether connect fails fast or slow.
The same random-port-no-collision-check pattern appears elsewhere in this file; worth auditing the other servers while in here.
asan + ubsan (full suite)failed on PR #759 with two checks that have nothing to do with that PR:Not deterministic: the same commit passed that job on its previous run, and locally
make asan-http && bash tests/test_http_server.shis 37/37. Both symptoms are empty responses (000,''), i.e. the second server was never reachable — not a budget-logic failure. The harness misdiagnoses its own setup failure as a DoS-gate regression, which is the shape that wastes the most time when it fires in CI.Three defects in the
HS27/HS28block (tests/test_http_server.sh:605-646), each measured on the dev box:1. The listen port is drawn from inside the kernel's ephemeral range.
The whole 50000-59999 window sits inside the ephemeral range, and this suite makes dozens of
curlconnections that each take an ephemeral port from it. The test can therefore lose the bind to its own client traffic. There is no collision check and no retry.2. The readiness loop's real budget is ~3.6s, not the ~33s it looks like.
--max-time 1only bounds the case where something is listening; a refused connection returns instantly, so this is effectively30 × sleep 0.1:For calibration, an
asan-httpserver on this idle box is ready in0.116s— a 31× margin, so slow start is not the likely trigger here, but the budget is far tighter than it reads.3. Neither failure is detected. If all 30 probes fail the loop just ends and the test proceeds against a server that isn't there. A
bindfailure in the server process goes to/tmp/eigs_http_srv2_$$.log, which is never inspected and never uploaded as a CI artifact — so the log that would name the real cause is discarded at the moment it becomes interesting.Suggested fix
failwith "server never came up" and dump/tmp/eigs_http_srv2_$$.log— a setup failure should say so instead of blaming the budget.The same random-port-no-collision-check pattern appears elsewhere in this file; worth auditing the other servers while in here.