Flaky test: `test_event_tape_replay_after_terminal` — final event races the terminal status row

[helebest]

Summary

tests/server/test_ingest_task.py::test_event_tape_replay_after_terminal is a
recurring CI timing flake. It fails intermittently with:

>       assert events[-1]["type"] == "final"
E       AssertionError: assert 'progress' == 'final'
tests/server/test_ingest_task.py:94: AssertionError

It is not tied to a specific code change — it flakes on unrelated diffs
(e.g. doc/version-only PRs) and lands on a different Python matrix leg
(3.12 vs 3.13) between reruns, so a rerun reliably turns it green. During the
0.6.5 release cycle alone it fired three times.

Root cause (test-side race)

The test trusts the task status row as a proxy for "the final event is on
the event tape", but the two are committed independently with no ordering
guarantee:

1. wait_task_terminal (tests/server/conftest.py:144) polls
   GET /v1/tasks/{id} until status ∈ {succeeded, failed, cancelled} and
   returns as soon as the status row is terminal.
2. The test then immediately reads the tape with no long-poll:
   GET /v1/tasks/{id}/events?from_seq=0&limit=1000&wait=0
   (tests/server/test_ingest_task.py:86-89).
3. It asserts events[-1]["type"] == "final".

When the runner flips the status row to terminal slightly before the final
event is appended/committed to the tape, the wait=0 read returns a tape whose
last entry is still the trailing progress event → the assertion sees
'progress' instead of 'final'.

Evidence (0.6.5 cycle)

• lint-type-test (3.12) on the feat(eval): surface absolute relevance scores in eval rows for OOD calibration (#249) #253 merge push — https://github.com/OpenDIKW/dikw-core/actions/runs/28324752190 (failed, then green on rerun)
• lint-type-test (3.13) on PR chore(release): prepare 0.6.5 #254 — https://github.com/OpenDIKW/dikw-core/actions/runs/28326041212 (failed, then green on rerun)

Related

A sibling task-subsystem timing flake has been observed too:
test_task_span_marks_cancel_not_error (intermittent ~30s terminal-state
timeout). Likely the same family of status-row-vs-event-tape / terminal-state
visibility races; worth fixing together.

Proposed direction

• Test-side (primary): stop trusting the status row as a tape proxy. Either
  long-poll the events endpoint (wait>0) or poll the tape until
  events[-1]["type"] == "final" before asserting — i.e. wait for the signal
  the test actually depends on. This removes the flake without touching product
  code.
• Product-side (open question for a maintainer): decide whether the terminal
  status row and the final event should be observable atomically (or the
  final event committed before the status flips), so that "status is terminal"
  is a sound guarantee that "the tape is complete" for any external consumer —
  not just this test. If yes, that is a small task-subsystem change with its own
  test.

Workaround until fixed

Rerun the failed CI job. It is a known non-blocking flake unless the PR diff
touches the task subsystem (server/tasks/**, the events/ingest routes).