Skip to content

Fix: preflight remote L3 timeouts before any startup resource - #1426

Merged
ChaoWao merged 1 commit into
hw-native-sys:mainfrom
ChaoWao:feat/p0.3c-1-remote-timeout-preflight
Jul 21, 2026
Merged

Fix: preflight remote L3 timeouts before any startup resource#1426
ChaoWao merged 1 commit into
hw-native-sys:mainfrom
ChaoWao:feat/p0.3c-1-remote-timeout-preflight

Conversation

@ChaoWao

@ChaoWao ChaoWao commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator

Scope

Narrow follow-up to #1420 (single-root startup budget). Closes the numeric
timeout preflight
ordering gap only: an invalid remote timeout must fail
before any startup resource exists, not after the subtree is built and rolled
back.

Before this PR the numeric values were validated, but too late:

  • Parent checked remote_session_timeout_s only in
    _activate_remote_sessions — which runs after _init_hierarchical already
    allocated mailbox shm, constructed the pre-fork _Worker mmap, and forked
    children. An invalid value built the whole subtree, then unwound it.
  • Daemon (_start_session) validated startup_remaining_s before the
    spawn, but not session_timeout_s: a valid startup_remaining_s plus an
    invalid session_timeout_s launched the runner, which only then died on it.

Delivered

  • Parent preflight. remote_session_timeout_s is validated at the top of
    _init_hierarchical, before any mailbox shm / pre-fork _Worker / child fork
    / daemon socket is created. The check is gated on the presence of remote
    workers
    (self._remote_worker_specs) — only a level >= 4 parent can carry
    them — so a worker with no remotes never validates an unused timeout;
    _activate_remote_sessions early-returns on the same no-remote path.
  • Daemon preflight. _start_session validates session_timeout_s (as well
    as the already-checked startup_remaining_s) before the ready pipe, manifest
    tempfile, and runner Popen.
  • Runner-side second validation unchanged (remote_l3_session).
  • Illegal 0 / negative / inf / nan now fail before resource creation on
    both paths.

Explicitly NOT in scope

  • No C++ / binding / wire / manifest-field changes.
  • No daemon reply / runner recycling changes.
  • No P0.2 admission / cancel / journal work.
  • Does not claim to validate the whole RemoteWorkerSpec — only the numeric
    timeout preflight.
  • Does not close the "healthy long task misjudged by command timeout"
    known issue.

Testing

  • Device-free UT (test_remote_startup_budget.py, 45 passed):
    TestParentPreflightBeforeResources and TestDaemonPreflightBeforeSpawn
    mock the resource constructors and assert zero SharedMemory / _Worker
    (parent) and os.pipe / NamedTemporaryFile / Popen (daemon) calls on
    illegal input, plus that a remote-less worker never validates the timeout.
    Verified failing against pre-change source, passing after.
  • tests/ut/py/test_worker/ + test_remote_l3_lifecycle.py: 276 passed,
    2 skipped — no regression from the now-earlier, remote-gated validation.
  • ruff + pyright clean on all touched files.

@coderabbitai

coderabbitai Bot commented Jul 21, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Startup timeout validation now occurs before hierarchical worker resources, daemon pipes, temporary files, or runner processes are created. Tests verify invalid parent and manifest timeout configurations fail during preflight.

Changes

Startup timeout preflight

Layer / File(s) Summary
Parent worker preflight
python/simpler/worker.py
Hierarchical initialization validates remote_session_timeout_s before creating shared memory, workers, child processes, or remote sessions.
Daemon spawn preflight
python/simpler/remote_l3_worker.py, tests/ut/py/test_worker/test_remote_startup_budget.py
_start_session validates manifest timeouts before resource creation or runner spawning; tests cover invalid session and startup-remaining values.
Estimated code review effort: 2 (Simple) ~10 minutes

Possibly related PRs

Poem

A rabbit checks the timeout gate,
Before pipes and runners wake.
No shared memory hops the fence,
Bad clocks stop at preflight hence.
Clean starts bound through every lane!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main change: earlier preflight validation of remote L3 timeouts before startup resources are created.
Description check ✅ Passed The description is directly aligned with the changeset and explains the parent and daemon timeout preflight updates and tests.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces early validation for remote session timeouts and startup budgets in both the parent worker and the remote L3 worker daemon, ensuring that invalid configurations fail fast before any system resources (such as shared memory, pipes, or subprocesses) are allocated. Unit tests have been added to verify this preflight behavior. The reviewer suggests wrapping the parent worker's timeout validation in a level check (self.level >= 4) to avoid redundant validation on level 3 workers.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread python/simpler/worker.py Outdated
@ChaoWao
ChaoWao force-pushed the feat/p0.3c-1-remote-timeout-preflight branch from a26236b to a48a4c5 Compare July 21, 2026 13:01
An invalid remote_session_timeout_s / session_timeout_s / startup_remaining_s
was rejected only after the parent had already built its subtree and the
daemon had already spawned the runner — the failure then had to unwind
partially-created resources instead of never creating them.

- Parent: validate remote_session_timeout_s at the top of _init_hierarchical,
  before any mailbox shm, pre-fork _Worker mmap, child fork, or daemon socket
  is created (previously only checked in _activate_remote_sessions, after the
  subtree was built). The check is gated on the presence of remote workers —
  only a level >= 4 parent can carry them — so a worker with no remotes never
  validates an unused timeout. _activate_remote_sessions early-returns on the
  no-remote path for the same reason.
- Daemon: validate session_timeout_s in _start_session before the ready pipe,
  manifest tempfile, and runner Popen — a valid startup_remaining_s no longer
  shields an invalid session_timeout_s from the pre-spawn gate.
- Runner-side (remote_l3_session) second validation is unchanged.
- Device-free UT proves zero resource-constructor calls on illegal
  0 / negative / inf / nan input on both the parent and daemon paths, and that
  a remote-less worker does not validate the remote timeout.
@ChaoWao
ChaoWao merged commit 065069d into hw-native-sys:main Jul 21, 2026
58 of 61 checks passed
@ChaoWao
ChaoWao deleted the feat/p0.3c-1-remote-timeout-preflight branch July 21, 2026 13:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant