Skip to content

test(eval): harden relay descendant cleanup against process-group timing #3770

Description

@liugddx

Problem

The test job for #3722 failed in the unrelated eval lifecycle suite at packages/eval/harbor/test_relay_lifecycle.py:648:

RelayLifecycleTest.test_settle_kills_descendants_before_verification_boundary
self.assertFalse(late_write.exists())
AssertionError: late_write remained present

The test starts a setsid --wait process whose parent exits while a forked descendant writes late-write after 300 ms. The relay should quiesce the entire process group before the verifier observes the workspace, but this run observed the delayed write.

Failure evidence: https://github.com/apache/maka/actions/runs/32754131081/job/97517633860

Scope

This is pre-existing and unrelated to #3722:

Suspected area

packages/eval/harbor/relay_agent.py uses the scope PID file and kill -0 -- -<pgid> to decide whether a process group is active before sending TERM/KILL. The failure suggests a timing or process-group observability gap: the scope was treated as inactive before the descendant's delayed write, so no effective termination was observed. The exact pgid/signal trace is not currently logged.

Suggested follow-up

  1. Add raw scope PID, process-group liveness, signal return code, and post-signal verification evidence.
  2. Make descendant quiescence robust when the group leader exits before the descendant.
  3. Keep the regression test deterministic on the Linux CI runner and verify repeated runs.

This issue should be tracked separately from the Windows sandbox lifecycle work in #3722.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions