Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,12 @@ $(BUILD_DIR)/test-pthread: tests/test-pthread.c | $(BUILD_DIR)
@echo " CROSS $< (with -lpthread)"
$(Q)$(CROSS_COMPILE)gcc -D_GNU_SOURCE -static -O2 -o $@ $< -lpthread

# test-process-lifecycle creates a worker to verify that process PIDs and
# thread TIDs share one namespace-wide allocator across fork children.
$(BUILD_DIR)/test-process-lifecycle: tests/test-process-lifecycle.c src/utils.h | $(BUILD_DIR)
@echo " CROSS $< (with -lpthread)"
$(Q)$(CROSS_COMPILE)gcc -D_GNU_SOURCE -static -O2 -Isrc -o $@ $< -lpthread

# test-thread-churn creates >64 threads to force thread-table slot reuse.
$(BUILD_DIR)/test-thread-churn: tests/test-thread-churn.c | $(BUILD_DIR)
@echo " CROSS $< (with -lpthread)"
Expand Down
50 changes: 46 additions & 4 deletions docs/internals.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,10 @@ Apple HVF imposes a handful of constraints that shape the rest of the design:
handled by the HVC #12 system-instruction trap path.
- Only `HV_SYS_REG_*` constants from Hypervisor.framework may be used for
register IDs.
- `hv_vcpu_t` value zero is a valid handle (normally the first vCPU in a VM),
not an invalid sentinel. `guest_t` and `thread_entry_t` therefore track
handle ownership with a separate `vcpu_valid` flag; signal preemption,
quiesce, ptrace, and teardown must consult that flag before calling HVF.
- Cross-thread vCPU register access is unreliable; all register access must
happen on the owning thread. This drives the snapshot protocol used by
both ptrace and the GDB stub.
Expand Down Expand Up @@ -462,9 +466,9 @@ are supported per VM.

`src/runtime/thread.c` and `thread.h`:

- `thread_entry_t` per thread holds the vCPU handle, host pthread, per-thread
signal mask, `clear_child_tid` (for `CLONE_CHILD_CLEARTID`), and the
thread's `SP_EL1` exception stack.
- `thread_entry_t` per thread holds the vCPU handle plus its explicit validity,
host pthread, per-thread signal mask, `clear_child_tid` (for
`CLONE_CHILD_CLEARTID`), and the thread's `SP_EL1` exception stack.
- `_Thread_local current_thread` gives O(1) access from syscall handlers.
- Each thread receives a 4 KiB EL1 exception stack carved out of the shim
data region.
Expand Down Expand Up @@ -541,7 +545,45 @@ state transfer:
4. Child receives state, creates its own VM, restores registers directly
into EL0 (bypassing the shim `_start` so callee-saved GPRs survive), and
enters the vCPU loop with `X0 = 0` (the child return from `clone`).
5. Parent records the child in the process table and returns the child PID.
5. Before spawning, the parent allocates a namespace-wide guest PID and
reserves both a local process-table slot and shared lifecycle entry. After
IPC succeeds it commits the child's host PID, releases the child into guest
code, and returns the child PID. Allocation or admission failure returns
`EAGAIN` without allowing an untracked child to run.

### Process Lifecycle And Guest PID 1

The first guest process in an elfuse invocation has guest PID 1 and therefore
becomes the fallback parent for orphaned descendants when no living child
subreaper is closer. elfuse does not insert a hidden init process and does not
automatically discard an adopted child's exit status merely because its new
parent is PID 1. As on Linux, the new parent must consume that status with a
`wait*()` call, or explicitly select no-zombie semantics with
`SIGCHLD = SIG_IGN` or `SA_NOCLDWAIT`.

The invocation-scoped lifecycle registry keeps the Linux wait-format terminal
status (including signal and core-dump bits) and the exiting process's resource
usage until the new parent consumes it. A parent already blocked in `wait*()`
periodically imports newly adopted descendants; adoption of an already exited
child also sends `SIGCHLD` to wake the adopter. Both the per-process wait table
and the invocation-wide lifecycle registry grow geometrically with the actual
fork-family population; the on-disk registry serializes only its live records.
An empty newly created registry is initialized on first use; a nonempty record
set that cannot be read or validated fails closed instead of being overwritten
as empty. Fork admission is reserved before the host helper starts, so
allocation or registry failure fails the new fork instead of silently dropping
a waitable child. Pre-spawn registry reservations are not imported as adopted
children, and a matching local reserved slot remains authoritative through the
registry-publish/local-commit window.

An application runtime used directly as guest PID 1 may not perform that
reaper role. In that case, adopted terminal statuses remain in elfuse's
invocation-scoped lifecycle registry, and a direct macOS child of the PID 1
elfuse process may also remain a host zombie until the guest waits, changes its
SIGCHLD disposition, or exits. Guest waitability and host-process cleanup are
separate concerns: a future host-only reaper could collect the macOS process
while retaining its exit status for a later guest wait, but no such background
reaper is currently provided.

### CoW Fork Path

Expand Down
1 change: 1 addition & 0 deletions src/core/bootstrap.c
Original file line number Diff line number Diff line change
Expand Up @@ -673,6 +673,7 @@ int guest_bootstrap_create_vcpu(guest_t *g,
HV_CHECK(hv_vcpu_create(&vcpu, &vexit, NULL));
startup_trace_step("hv_vcpu_create", t0);
g->vcpu = vcpu;
g->vcpu_valid = true;
g->exit = vexit;
*out_vcpu = vcpu;
*out_vexit = vexit;
Expand Down
8 changes: 4 additions & 4 deletions src/core/guest.c
Original file line number Diff line number Diff line change
Expand Up @@ -669,11 +669,11 @@ void guest_destroy(guest_t *g)
* VM. This prevents hv_vm_destroy from racing with active vCPUs that may
* still be running if thread join timed out during exit_group.
*/
thread_destroy_all_vcpus();
if (g->vcpu) {
bool main_vcpu_destroyed = thread_destroy_all_vcpus(g->vcpu, g->vcpu_valid);
if (g->vcpu_valid && !main_vcpu_destroyed)
hv_vcpu_destroy(g->vcpu);
g->vcpu = 0;
}
g->vcpu_valid = false;
g->vcpu = 0;
/* Unmap each HVF segment. hv_vm_destroy releases all stage-2 state
* regardless, but unmapping explicitly keeps invariants clean for
* downstream tools (Instruments, leak detectors).
Expand Down
2 changes: 2 additions & 0 deletions src/core/guest.h
Original file line number Diff line number Diff line change
Expand Up @@ -450,6 +450,8 @@ typedef struct {
uint64_t ttbr0; /* TTBR0 value (IPA of L0 page table) */
uint64_t ttbr1; /* TTBR1 value (IPA of L0 kernel page table; 0 if unused) */
hv_vcpu_t vcpu; /* vCPU handle */
bool vcpu_valid; /* Handle value zero is valid; track ownership separately.
*/
hv_vcpu_exit_t *exit; /* vCPU exit info */
uint32_t ipa_bits; /* IPA bits requested from HVF */

Expand Down
2 changes: 1 addition & 1 deletion src/main.c
Original file line number Diff line number Diff line change
Expand Up @@ -656,7 +656,7 @@ int main(int argc, char **argv)

/* vcpu_run_loop owns guest execution until exit, fatal signal, or timeout.
*/
exit_code = vcpu_run_loop(vcpu, vexit, &g, verbose, timeout_sec);
exit_code = vcpu_run_loop(vcpu, vexit, &g, verbose, timeout_sec, NULL);

/* Tear down debugger state before joining workers: a worker parked in
* gdb_stub_handle_stop() stays active (not deactivated) until this
Expand Down
Loading
Loading