Add Graph Execution to host_build_graph - #1444
Open
TaoZQY wants to merge 1 commit into
Open
Conversation
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
📝 WalkthroughWalkthroughAdds host-side graph capture and replay for ChangesGraph Execution Runtime
Estimated code review effort: 5 (Critical) | ~120 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
TaoZQY
force-pushed
the
feat/graph-execution
branch
2 times, most recently
from
July 22, 2026 12:36
fd8fc05 to
66f183e
Compare
- Cache repeated task DAGs with saved topology and dynamic TaskArgs sources - Submit one Graph task on cache hits and expand nodes in the device Scheduler - Reuse compact host submissions and graph-affine AICPU execution blocks - Add Graph/Host-Orchestrator DFX lanes, simulation coverage, and Qwen API docs - Fix predicate-field compatibility in a2a3/a5 runtime builds Co-authored-by: Crane-Liu <c.wliu@outlook.com>
ChaoWao
added a commit
to ChaoWao/simpler-fork
that referenced
this pull request
Jul 26, 2026
A forked sub/child/chip worker never checked whether its parent was still alive. When the parent died without `Worker.close()` — SIGKILL from a `timeout`, an OOM kill, a cancelled CI job, an editor killing a wedged pytest — every child was reparented and kept polling a mailbox nobody would ever write to. Because that poll is an unbounded busy-wait, each survivor held a full core indefinitely. This is not theoretical. On a shared dev box six such processes belonging to two different users had been alive 5-7 days each at ~99.6% CPU, six cores permanently consumed. Four of the six carried `tests/ut/py/test_worker/test_startup_readiness.py` on their command line: runs whose parent was killed rather than allowed to finish. `_run_mailbox_loop` now samples `getppid()` on its idle path and leaves by the same SHUTDOWN route once it changes, so a child tears down exactly as it would on a clean shutdown. Compare against the pid captured at loop entry rather than testing for pid 1: a subreaper (container init, a systemd user session) adopts orphans instead of init, so the pid changes but never becomes 1. A live parent's pid cannot change, so the check cannot fire spuriously. Sampling every `_PARENT_LIVENESS_POLL_INTERVAL` idle polls puts a `getppid()` roughly every 100 us against a ~0.1 us poll — fast enough that an orphan goes away before it is noticeable, cheap enough to be lost in the noise of the poll itself. Residual gap: a parent that dies between `os.fork()` and the child reaching the loop is already gone when the pid is captured, so that child is not detected. Closing it needs the pid handed in from before the fork, or `PR_SET_PDEATHSIG`, which is Linux-only and therefore an optimisation on top of this rather than the mechanism. `test_orphan_child_reaping.py` SIGKILLs a subprocess parent and asserts every child is gone within 20 s. Three details are load-bearing: - It reads the pids from the Worker's own `_sub_pids` rather than `pgrep -P`. Enumerating children externally also catches the transient shell that runs pgrep, and inside a container's shallow pid namespace it picks up unrelated low pids: on CI that produced "1 of 6 children outlived ... [3]", waiting on and then SIGKILLing a container process that was never ours. - It writes to a file and never reads a pipe. Orphans inherit the parent's stdout, so on regression they hold a pipe's write end open and reading one would hang the test for as long as the bug survives instead of failing it. - Its `finally` kills whatever survived, so a regression does not hand spinning processes to the next test. It runs on macOS as well as Linux: reparenting on parent death is POSIX behaviour, and nothing here is platform-specific once the pid list comes from the Worker. Also drop `src/a2a3/runtime/host_build_graph/docs/GRAPH_EXECUTION.md`, which hw-native-sys#1494 added by accident: it is documentation from the open hw-native-sys#1444 (`feat/graph-execution`) that happened to be sitting untracked in the working tree, and a `git add -A` swept it into that commit. Removing it restores main to the state hw-native-sys#1444 expects to merge into. It is unrelated to everything else here and can be reviewed independently of the fix. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ChaoWao
added a commit
to ChaoWao/simpler-fork
that referenced
this pull request
Jul 26, 2026
A forked sub/child/chip worker never checked whether its parent was still alive. When the parent died without `Worker.close()` — SIGKILL from a `timeout`, an OOM kill, a cancelled CI job, an editor killing a wedged pytest — every child was reparented and kept polling a mailbox nobody would ever write to. Because that poll is an unbounded busy-wait, each survivor held a full core indefinitely. This is not theoretical. On a shared dev box six such processes belonging to two different users had been alive 5-7 days each at ~99.6% CPU, six cores permanently consumed. Four of the six carried `tests/ut/py/test_worker/test_startup_readiness.py` on their command line: runs whose parent was killed rather than allowed to finish. `_run_mailbox_loop` now samples `getppid()` on its idle path and leaves by the same SHUTDOWN route once it changes, so a child tears down exactly as it would on a clean shutdown. Compare against the pid captured at loop entry rather than testing for pid 1: a subreaper (container init, a systemd user session) adopts orphans instead of init, so the pid changes but never becomes 1. A live parent's pid cannot change, so the check cannot fire spuriously. Sampling every `_PARENT_LIVENESS_POLL_INTERVAL` idle polls puts a `getppid()` roughly every 100 us against a ~0.1 us poll — fast enough that an orphan goes away before it is noticeable, cheap enough to be lost in the noise of the poll itself. Residual gap: a parent that dies between `os.fork()` and the child reaching the loop is already gone when the pid is captured, so that child is not detected. Closing it needs the pid handed in from before the fork, or `PR_SET_PDEATHSIG`, which is Linux-only and therefore an optimisation on top of this rather than the mechanism. `test_orphan_child_reaping.py` SIGKILLs a subprocess parent and asserts every child is gone within 20 s. Three details are load-bearing: - It reads the pids from the Worker's own `_sub_pids` rather than `pgrep -P`. Enumerating children externally also catches the transient shell that runs pgrep, and inside a container's shallow pid namespace it picks up unrelated low pids: on CI that produced "1 of 6 children outlived ... [3]", waiting on and then SIGKILLing a container process that was never ours. - It writes to a file and never reads a pipe. Orphans inherit the parent's stdout, so on regression they hold a pipe's write end open and reading one would hang the test for as long as the bug survives instead of failing it. - Its `finally` kills whatever survived, so a regression does not hand spinning processes to the next test. - It asserts the parent died from its own SIGKILL. A parent that fails before forking otherwise surfaces as an unexplained "expected 3 pids, got []", which reads like a defect in the code under test rather than an environment problem in the harness. It runs on macOS as well as Linux: reparenting on parent death is POSIX behaviour, and nothing here is platform-specific once the pid list comes from the Worker. Also drop `src/a2a3/runtime/host_build_graph/docs/GRAPH_EXECUTION.md`, which hw-native-sys#1494 added by accident: it is documentation from the open hw-native-sys#1444 (`feat/graph-execution`) that happened to be sitting untracked in the working tree, and a `git add -A` swept it into that commit. Removing it restores main to the state hw-native-sys#1444 expects to merge into. It is unrelated to everything else here and can be reviewed independently of the fix. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ChaoWao
added a commit
that referenced
this pull request
Jul 26, 2026
A forked sub/child/chip worker never checked whether its parent was still alive. When the parent died without `Worker.close()` — SIGKILL from a `timeout`, an OOM kill, a cancelled CI job, an editor killing a wedged pytest — every child was reparented and kept polling a mailbox nobody would ever write to. Because that poll is an unbounded busy-wait, each survivor held a full core indefinitely. This is not theoretical. On a shared dev box six such processes belonging to two different users had been alive 5-7 days each at ~99.6% CPU, six cores permanently consumed. Four of the six carried `tests/ut/py/test_worker/test_startup_readiness.py` on their command line: runs whose parent was killed rather than allowed to finish. `_run_mailbox_loop` now samples `getppid()` on its idle path and leaves by the same SHUTDOWN route once it changes, so a child tears down exactly as it would on a clean shutdown. Compare against the pid captured at loop entry rather than testing for pid 1: a subreaper (container init, a systemd user session) adopts orphans instead of init, so the pid changes but never becomes 1. A live parent's pid cannot change, so the check cannot fire spuriously. Sampling every `_PARENT_LIVENESS_POLL_INTERVAL` idle polls puts a `getppid()` roughly every 100 us against a ~0.1 us poll — fast enough that an orphan goes away before it is noticeable, cheap enough to be lost in the noise of the poll itself. Residual gap: a parent that dies between `os.fork()` and the child reaching the loop is already gone when the pid is captured, so that child is not detected. Closing it needs the pid handed in from before the fork, or `PR_SET_PDEATHSIG`, which is Linux-only and therefore an optimisation on top of this rather than the mechanism. `test_orphan_child_reaping.py` SIGKILLs a subprocess parent and asserts every child is gone within 20 s. Three details are load-bearing: - It reads the pids from the Worker's own `_sub_pids` rather than `pgrep -P`. Enumerating children externally also catches the transient shell that runs pgrep, and inside a container's shallow pid namespace it picks up unrelated low pids: on CI that produced "1 of 6 children outlived ... [3]", waiting on and then SIGKILLing a container process that was never ours. - It writes to a file and never reads a pipe. Orphans inherit the parent's stdout, so on regression they hold a pipe's write end open and reading one would hang the test for as long as the bug survives instead of failing it. - Its `finally` kills whatever survived, so a regression does not hand spinning processes to the next test. - It asserts the parent died from its own SIGKILL. A parent that fails before forking otherwise surfaces as an unexplained "expected 3 pids, got []", which reads like a defect in the code under test rather than an environment problem in the harness. It runs on macOS as well as Linux: reparenting on parent death is POSIX behaviour, and nothing here is platform-specific once the pid list comes from the Worker. Also drop `src/a2a3/runtime/host_build_graph/docs/GRAPH_EXECUTION.md`, which #1494 added by accident: it is documentation from the open #1444 (`feat/graph-execution`) that happened to be sitting untracked in the working tree, and a `git add -A` swept it into that commit. Removing it restores main to the state #1444 expects to merge into. It is unrelated to everything else here and can be reviewed independently of the fix
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
host_build_graphruntime;fanin/fanout, roots, argument sources, and static scalars;
GRAPHtask from the host Orchestrator on a cache hit, then letthe device Scheduler materialize and execute the saved topology;
L2TaskArgsAPI,without public GraphBindings, Patch, or GraphArgs types;
submission and graph-affine AICPU execution pools;
graph_prepareDFX lanes;Qwen3-14B-style upstream integration example.
Execution flow
ordinary task submits and save the complete DAG definition.
GRAPHdescriptor containing acompact definition and the current
L2TaskArgsvalues.per scheduling slice, activates the saved roots, and releases saved fanout
edges as nodes complete.
downstream dependencies retain normal task semantics.
Upstream API: Qwen decoder layer
The caller packages hidden state, weights, output,
layer_id, andtoken_positionintoL2TaskArgs; the wrapper does not repeat them in itssignature. The function pointer is the Graph ID for structurally identical
layers. Tensor addresses and scalar values may change; tensor shape, dtype,
stride, size, direction, and task topology must remain compatible. See
GRAPH_EXECUTION.mdfor the complete three-layer Qwen-style example and current limitations.
Scope
upstream/main;host_build_graph;tensormap_and_ringbufferchanges are API-plumbing compatibility and twopredicate-field build fixes, not Graph Execution enablement.
Testing
python simpler_setup/build_runtimes.py --platforms a2a3sim a5sim;a2a3sim:1 passed;24 passed;host_build_graphsimulation:12 passed, 1 deselected;task_submit, cache-hitgraph_submit, Schedulergraph_prepare, and Graph Execution lanes.