Fix Linux child lifecycle semantics

[Xalestar]

Summary

• allocate guest process IDs and thread IDs across one elfuse invocation's entire fork family instead of resetting the allocator in every host process
• retain zombie status and resource usage until a consuming wait, including repeatable waitid(WNOWAIT) observations
• deliver SIGCHLD from child exit and implement the explicit SIG_IGN / SA_NOCLDWAIT no-zombie dispositions
• track process lifecycle state across elfuse host processes so live and zombie orphans are adopted by the nearest living child subreaper, or guest PID 1
• reserve local and shared child bookkeeping before host spawn, failing fork() with EAGAIN instead of allowing an untrackable child to run
• prevent concurrent waits from importing a fork-admission reservation as a second child record
• grow per-process wait state and invocation-wide lifecycle state with the fork family while failing allocation and invalid-registry paths closed
• preserve Linux signal-termination wait status, including parent-directed SIGKILL of a compute-bound child
• add one portable 20-case lifecycle test shared by the elfuse and QEMU/Linux matrix runners

Design notes

elfuse implements each guest fork() with a separate macOS host process. Parent-local state is therefore insufficient for nested PID/TID allocation, retained zombie state, or orphan adoption.

This change adds invocation-scoped, flock-serialized PID and lifecycle registries in the per-user private temporary directory. The lifecycle registry retains guest PID/PPID/PGID, subreaper state, Linux wait-format terminal status, and resource usage until a consuming guest wait removes the entry. Shared PID/TID allocation is fail-closed: path, open, lock, read, or write failure returns EAGAIN rather than falling back to an unsynchronized process-local counter.

Live reparenting uses the existing cross-process control transport; a registry pending/acknowledgment handshake and a post-bootstrap authoritative sync close the window where the fork header's original PPID could otherwise overwrite a newly selected adopter. When an already-terminal child is reparented, the adopter receives SIGCHLD; blocking wait4 and waitid also re-import adopted children during their retry loops and before returning ECHILD.

The shared ID allocator is used by process-style fork(), clone(CLONE_THREAD), and waitable clone(CLONE_VM) children. This keeps live process PIDs and thread TIDs in one invocation-wide ID space; the lifecycle registry itself continues to track process children rather than treating CLONE_THREAD workers as orphanable/waitable processes.

Fork admission is transactional. The parent reserves both a local process-table slot and a shared lifecycle entry before spawning the host helper. The child remains behind an admission byte after IPC initialization and enters guest code only after parent-side registration commits. Pre-spawn registry reservations are excluded from adopted-child import, and the matching local reserved slot remains authoritative after the host PID is published but before local commit. This prevents a concurrent wait from creating a phantom second record for the same child.

Both the per-process wait table and the invocation-wide lifecycle registry begin with small allocations and grow geometrically with the fork-family population under their respective locks. The on-disk lifecycle registry serializes only its live records. A newly created zero-length registry is initialized on first use; a nonempty registry that cannot be read or validated fails closed instead of being overwritten as empty. Allocation or reservation failure is returned to the guest as EAGAIN.

Fatal guest signals store Linux signal/core wait bits rather than being folded into a normal exit code. Signal preemption and teardown track Hypervisor.framework vCPU ownership with a separate validity flag because handle value zero is valid; this allows a parent-directed signal to interrupt the first, compute-bound vCPU correctly.

Exit-time SIGCHLD delivery triggers targeted automatic reaping only for children already marked terminal in the lifecycle registry. Parent-side child registration preserves lifecycle state that the child or a reparent transaction published first. waitid(P_PGID, ...) and the auto-reap wait path use process-group-aware matching, and a no-event waitid(WNOHANG) clears the complete 128-byte guest siginfo_t.

Testing

Final validation at commit 847ccb9:

• targeted lifecycle test under elfuse: 20 passed, 0 failed
• targeted lifecycle test under QEMU/Linux: 20 passed, 0 failed
• make -B check: all 69 internal tests passed; BusyBox reported 82 passed, 0 failed, 2 skipped
• make test-matrix-elfuse-aarch64: 241 passed, 0 failed, 4 skipped
• make lint: exit 0, existing warnings only
• .ci/check-format.sh and git diff --check: passed
• GitHub CI: 10/10 checks passed, including Build, Lint, Infer, clang-tidy, scan-build, Release, ASAN, TSAN, UBSAN, and Cubic

Additional stress and sanitizer validation performed while hardening the lifecycle implementation:

• 100 consecutive host-level lifecycle runs: 100/100 passed, with all 20 cases passing each run
• forced TSAN rebuild: sanitizer subset 48/48 passed; the lifecycle test also passed 20/20 under the TSAN-linked elfuse binary

The portable lifecycle cases cover nested process PID uniqueness; process/thread PID-TID uniqueness; WNOHANG; repeated WNOWAIT; normal and auto-reap waitid(P_PGID) matching; complete no-event siginfo_t clearing; normal-exit versus parent-directed signal status; concurrent wait/fork admission without duplicate records; delayed and reverse-order zombie reaping; 65 retained zombies; explicit no-zombie SIGCHLD dispositions; exit-time SIGCHLD delivery; PID 1 reparenting; live/zombie child-subreaper adoption; a bounded non-reaping PID 1 case; blocking-wait adoption wakeup; and adopted signal status/resource-usage retention.

Out of scope / follow-ups

• No hidden PID 1 reaper: elfuse does not insert an init process or automatically discard adopted exit statuses merely because their new parent is guest PID 1. As on Linux, guest PID 1 must call wait*() or explicitly select no-zombie semantics with SIGCHLD = SIG_IGN / SA_NOCLDWAIT. An application runtime used directly as guest PID 1 may therefore retain adopted statuses indefinitely, and a direct macOS child may remain a host zombie until the guest waits, changes its SIGCHLD disposition, or exits. A future host-only reaper could collect macOS children while preserving guest-visible status.
• PR_SET_PDEATHSIG support: this opt-in parent-death API is not required for the zombie retention, wait ownership, orphan reparenting, or child-subreaper semantics fixed here. It should be tested and implemented separately.
• Stopped orphaned process groups: the Linux/POSIX SIGHUP followed by SIGCONT rule remains a separate job-control follow-up.
• Existing-zombie flush on an explicit SIG_IGN transition: changing SIGCHLD from SIG_DFL to explicit SIG_IGN does not immediately flush already-terminal children; they are removed by the next wait path. This is lower-priority follow-up work.
• Bare-signal fallback parsing: the fallback parser does not reset errno/ERANGE exactly like the structured parser. An overflowed token is dropped; this is an inconsistency, not an exploitable routing issue.
• Registry scaling: lifecycle mutations serialize only live records, but still rewrite all of them under the global flock. This remains an O(n) scaling limit rather than a correctness failure.

[Max042004]

Reparenting an orphan to guest PID 1 is not sufficient on its own: the adopted child is only imported and consumed when the new parent calls wait*(), or explicitly opts into SIGCHLD=SIG_IGN / SA_NOCLDWAIT. In the common case, guest PID 1 is an application runtime (JVM, Python, Node.js, etc.), not an init process, and it may never reap adopted descendants.

As a result, this change can retain orphan exit statuses indefinitely (and may retain host zombies for direct children) even though the PPID has been updated. We need either an elfuse-managed init/reaper for guest PID 1, automatic reaping semantics for adopted children, or a clearly documented limitation. Please add a regression test where a non-reaping guest PID 1 adopts an orphan that exits.

If choose elfuse-managed init/reaper process, it necessary to measure how much cost is introduced in startup time.

[Xalestar]

Thanks — agreed that this needed explicit regression coverage and documentation.

I chose the documented-limitation path rather than automatically discarding guest-visible exit status. Commit 3d7bee adds a regression test for the non-reaping guest PID 1 case (O-06 in test-process-lifecycle.c), where:

• a leaf becomes orphaned and is adopted by guest PID 1;
• PID 1 deliberately performs no consuming wait for a bounded interval;
• two waitid(P_PID, ..., WEXITED | WNOHANG | WNOWAIT) calls observe the same PID and exit status;
• a final waitpid() consumes the status only to clean up the test.

In the QEMU/Linux reference lane, the test process is normally not PID 1, so it verifies that system init owns the orphan and that the former ancestor receives ECHILD.

docs/internals.md and the PR description now explicitly document that elfuse does not insert a hidden init/reaper. An application used directly as guest PID 1 must call wait*() or select SIGCHLD=SIG_IGN / SA_NOCLDWAIT. A direct macOS child may otherwise remain a host zombie.

Automatically discarding the guest status solely because the new parent is PID 1 would change Linux wait semantics. A host-only reaper that collects the macOS process while retaining guest-visible status remains possible as separate hardening. Since this PR does not add a managed init/reaper, it introduces no associated startup-time cost.

Local validation:

• targeted lifecycle test: 16 passed, 0 failed
• 100/100 stress runs, all 16 cases passing each run
• make check
• make lint (exit 0; existing warnings only)
• elfuse matrix: 240 passed, 0 failed, 4 skipped
• QEMU matrix: 219 passed, 0 failed, 25 skipped

[henrybear327]

Is TID in scope for this PR to also address?

[Xalestar]

@henrybear327 Yes — the narrow PID/TID allocation-uniqueness part is in scope.

proc_alloc_pid() is shared by process-style fork(), clone(CLONE_THREAD), and waitable clone(CLONE_VM), so moving it to an invocation-wide allocator also directly affects TIDs.

Commit 3d7bee adds a regression test for process/thread ID uniqueness (PID-02 in test-process-lifecycle.c). A fork child creates a live worker thread and verifies that the thread-group leader's TID equals its PID, while the worker TID does not collide with either the parent or child process PID. The same test passes under elfuse and QEMU/Linux.

This does not expand the PR into full CLONE_THREAD lifecycle or orphan/wait semantics; those remain process-child lifecycle concerns.

[jserv]

The cross-process flock read-modify-write discipline is sound. Findings below concern lifecycle edge cases. Three lower-priority items not worth their own anchor: installing SIG_IGN when the prior disposition was SIG_DFL doesn't discard existing zombies immediately (the flush in signal.c is gated on the old action, so zombies linger until the next wait*); sig_drain_cb's bare-signal fallback parse skips the errno/ERANGE reset the structured path does (an overflowed token just drops the record — not exploitable, but inconsistent); and lifecycle_save_locked rewrites the entire fixed registry under the global flock on every fork/setpgid/setsid/subreaper change, a scaling cliff rather than a bug.

[jserv]

lifecycle_import_children() runs once at the top of sys_wait4/sys_waitid, outside the retry loop, and proc_process_exit (:1261) sends SIGCHLD only to self->ppid while skipping reparented exited children (:1263). A subreaper blocked in wait4(-1) gets no signal for a zombie an unrelated exit reparents to it, and never re-imports it; if it has no live children it can return ECHILD while a waitable zombie for it sits in the registry. Re-import inside the wait loop, and send SIGCHLD to adopter_pid for each reparented exited child.

[Xalestar]

Fixed in the latest branch.

proc_process_exit() now sends SIGCHLD to the adopter for each already-terminal child being reparented. Both wait4() and waitid() also re-import lifecycle children inside their blocking retry paths and before returning ECHILD.

O-07 in test-process-lifecycle.c starts a blocking waitpid(-1, ...) before an intermediate branch exits, then verifies that the subreaper wakes and consumes the newly adopted zombie with its original status.

Validation at commit 1d11b88: elfuse 20/20, QEMU/Linux 20/20, full elfuse matrix 240 passed with 0 failed, and GitHub CI 10/10 including TSAN.

[jserv]

lifecycle_entry_t stores only exit_status, no struct rusage, and proc_process_exit (:1210) publishes only status. A direct parent host-reaps real rusage, but an adopted zombie read from this registry returns zero rusage — contradicting the "retain zombie status and resource usage" claim. Store rusage in the registry before the original host parent loses waitability, or narrow the documented behavior and add a test for adopted-zombie rusage.

[Xalestar]

Fixed in the latest branch.

The lifecycle registry now retains struct rusage, its validity flag, and the authoritative Linux wait-format status. proc_process_exit() snapshots RUSAGE_SELF, and importing an adopted terminal child preserves that usage for the adopter's consuming wait.

O-08 in test-process-lifecycle.c burns CPU, terminates the adopted child via SIGKILL, and verifies both WIFSIGNALED / SIGKILL and non-zero CPU usage after reparenting.

Validation at commit 1d11b88: elfuse 20/20, QEMU/Linux 20/20, full elfuse matrix 240 passed with 0 failed, and GitHub CI 10/10 including TSAN.

[jserv]

Rework the Git history by squashing related commits into fewer, more meaningful commits.

[henrybear327]

Confirmed that it improves the situation on the LTP conformance test!

[jserv]

Thank @Xalestar for contributing!