Skip to content

fix: post-merge review sweep — lint blind spots, bagel prefill guard, label workflows, refl nits - #290

Merged
haonan3 merged 9 commits into
mainfrom
fix/post-merge-review-sweep
Jul 31, 2026
Merged

fix: post-merge review sweep — lint blind spots, bagel prefill guard, label workflows, refl nits#290
haonan3 merged 9 commits into
mainfrom
fix/post-merge-review-sweep

Conversation

@haonan3

@haonan3 haonan3 commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

Summary

Fix batch from an adversarial post-merge review of the five recently merged PRs (#210, #277, #279, #284, #286). Every item below was reproduced/verified before fixing; the review evidence lives in the per-item notes.

Boundary lint had systematic blind spots (#279), and they already mattered on the tree:

  • The cross-package rule missed from experimental import <sibling>, bare import experimental, relative imports climbing past the package root, and treated top-level experimental/*.py (incl. __init__.py) as a lint-free shared space — the most natural spellings of exactly the sideways borrowing the README forbids. All caught now, in any spelling.
  • The "core dependency" set was [project.dependencies] only, so every extras pin — torch, torchvision, sglang, vllm — was re-declarable undetected; the in-tree videoalign/requirements.txt was already re-declaring torchvision with no signal. The set now unions base deps, every [project.optional-dependencies] extra, and [tool.uv] override-dependencies; the scan covers requirements*.txt and fails closed on pip option/include lines and URL/VCS lines it cannot vet. The videoalign re-pin is dropped (comment documents that the engine extra provides it) and the face extras file documents the same for its top-level torchvision import.
  • New tier-direction rule in check_recipe_targets.py: recipes outside experimental/ may not _target_ into experimental.*, and a package's recipes may not target a sibling package — _target_ is this repo's primary wiring mechanism and previously carried zero directional signal.
  • experimental/README.md now states which rules are script-enforced (1, 2, 4) vs review-enforced (3, 5, 6), and that private packages are lint-gated locally only.

Bagel (#277 follow-up): BagelPipeline._build_contexts runs the same vendored inference-signature prefills the #277 fix declared eval()-load-bearing, but with no guard — SFT/trainside callers reaching it in train() get the exact packed_query_sequence TypeError back. Same guard-and-restore applied.

#284 follow-ups: the two moved converters landed inside gitignored datasets/* (tracked but invisible to ruff/rg/editors; new sibling files needed add -f) — the dirs are whitelisted with their jsonl outputs kept local-only. Two recipe comments pointing at nonexistent scripts/*.sh launchers now point at examples/run_experiment_single_node.sh. experimental/private_* also covers stray private files now (was dir-only).

Label workflows (#286 follow-ups): merged/closed PRs shed their status label (they kept need review forever — see #210/#279 right now); synchronize self-heals a PR that lost its label to a missed event/transient failure; the review-labels consumer drops its shared concurrency group (GitHub keeps one pending run per group, so reviews landing on two PRs within seconds silently dropped the earlier sync).

refl nits (#210 follow-ups): seed: null no longer TypeErrors at the worker (int(params.seed) had no None branch; generate_latents already supports unseeded); the unreachable grad-input mismatch guard in worker.py is dropped (its dict-index comprehension KeyErrors first, and resolve() consumes the same keys earlier still).

Related Issue

N/A

Test Plan

  • Negative/positive battery for both guards (34 checks, all pass): every blind spot above rebuilt as a synthetic violation in a lab tree — from-import/bare/relative/__init__-hub sideways imports flagged; extras pin, uv-override pin, case-variant name, -r include, VCS line, and requirements-gpu.txt coverage flagged; self-imports (absolute/relative/from-bare), core imports, additive names, and name @ url direct refs stay clean; core→experimental and sibling _target_s flagged while own-package recipes pass with no unresolved noise.
  • Real tree: python lint/check_experimental_boundaries.py → ok; python lint/check_recipe_targets.py2310 recipe _target_ paths resolve. (unchanged count).
  • git check-ignore battery: both converters no longer ignored; their *.jsonl outputs, other datasets/* dirs, experimental/private_notes.py, and experimental/private_a/f.py all ignored.
  • python -m py_compile on all six edited .py files; bash -n on both edited workflow scripts (extracted from YAML).
  • SKIP=no-commit-to-branch pre-commit run --all-files → all 16 hooks pass (both hardened guards included).
  • Not run: GPU smoke for the bagel guard path (CPU box) — the change is mode-neutral by construction (guard-and-restore around an already-no_grad block, same pattern as the merged fix(bagel): rebuild replay KV contexts in eval() so navit keeps inference dispatch #277 fix); and the master_dtype: fp32 verification gap flagged for feat(experimental): add WAN ReFL as a self-contained training package #210 is deliberately NOT addressed here (owner action on the pinned-torch env, see Reviewer Notes).

Compatibility / Risk

  • The boundary/targets guards get strictly stricter; the current tree passes clean. Future violations that previously slipped through will now fail pre-commit/CI — that is the point.
  • Label workflows: behavior changes are label-lifecycle-only (closed cleanup, self-heal on synchronize, no shared concurrency). The pull_request_target consumers still check out nothing and interpolate no PR-controlled strings; the PR-number regex guard in the review consumer is untouched.
  • Bagel: guard-and-restore around prefills; freeze/rollout paths unaffected (colocate short-circuits before this call).
  • refl: seed: null now takes the supported unseeded path instead of crashing.

Reviewer Notes

Checklist

  • I reviewed the changed code and removed unrelated/generated artifacts.
  • I updated tests, docs, and configs where needed, or explained why not.

haonan3 added 6 commits July 31, 2026 19:34
… review

- cross-package rule now catches 'from experimental import <sibling>',
  bare 'import experimental', relative imports climbing past the package
  root, and imports from top-level experimental/*.py (no shared space)
- core-dependency set now unions [project.dependencies], every
  [project.optional-dependencies] extra, and [tool.uv]
  override-dependencies — extras pins (torch & co) no longer sail through
- requirements scan covers requirements*.txt, rejects pip option/include
  lines and unparseable URL/VCS requirements (fail closed)
- check_recipe_targets gains the tier-direction rule: recipes outside
  experimental/ may not target experimental.*, and a package's recipes
  may not target a sibling package
- reward extras requirements: drop the (now-flagged) bare torchvision
  re-pin in videoalign, document that face/videoalign get torch(vision)
  from the core stack; README states which rules are script-enforced
…lay path

BagelPipeline._build_contexts runs the same vendored inference-signature
prefills the #277 fix guarded in BagelDiffusionStage, but relied on
callers already being in eval(). SFT/trainside can reach it mid-training;
guard-and-restore instead of assuming the mode.
DiffusionSamplingParams.seed is Optional and generate_latents supports
None (unseeded path), but the call sites int()'d unconditionally.
The dict-index comprehension raises KeyError before the length check can
ever fire, and resolve() consumes the same keys earlier still.
…h comments

datasets/* ignored the two converters #284 moved in (tracked but hidden
from ruff/rg/editors; new siblings needed add -f) — whitelist the dirs,
keep their jsonl outputs local-only. experimental/private_* now also
covers stray private files. Two recipe comments pointed at scripts/*.sh
paths that no longer exist (runner lives at examples/).
…urrency

- closed/merged PRs shed their status label (they used to keep 'need
  review' forever)
- synchronize now applies 'need review' when a PR carries no status
  label at all, so one missed event no longer sticks forever
- drop the shared concurrency group on the review-labels consumer:
  GitHub keeps one pending run per group, silently dropping the earlier
  PR's sync when reviews land on two PRs within seconds
…recipes, no-space direct refs

- pr-review-status-labels: per-PR concurrency group (run-id fallback for
  fork PRs whose workflow_run.pull_requests is empty) restores the
  same-PR serialize-latest semantics the removed global group provided,
  without the cross-PR pending-slot drops; DELETE tolerates the 404 a
  parallel fork-PR run can leave behind
- check_recipe_targets: a recipe at the experimental/ root can no longer
  wire experimental.* — a recipe belongs inside one package
- check_experimental_boundaries: PEP 508 direct references without
  spaces (name@git+https://...) parse instead of erroring; bare URL/VCS
  lines still fail closed
@haonan3

haonan3 commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator Author

bca7e1e8 addresses an external (Codex) review round — all three findings verified real and fixed:

  1. Same-PR label race (P2): removing the global concurrency group traded cross-PR pending-slot drops for same-PR parallel runs racing label writes (and set -e dying on the 404 a parallel DELETE leaves). Now: per-PR group pr-review-status-labels-${{ workflow_run.pull_requests[0].number || run_id }} — same-repo PRs get back the serialize-latest semantics, fork PRs (empty pull_requests) fall back to per-run with a tolerant DELETE (|| true) absorbing the benign 404.
  2. Tier-root recipe bypass (P2): experimental/root_escape.yaml targeting experimental.refl.* slipped through the len(parts) > 2 sibling guard. A recipe at the experimental/ root now errors — a recipe belongs inside one package.
  3. No-space PEP 508 direct refs (P3): name@git+https://... was mis-flagged unparseable (only " @ " was recognized). @ joined the follower set — the name still gets extracted and checked against the core set (transformers@git+... correctly reports as a re-declared core dep), while bare git+/URL lines still fail closed. Kept the stdlib-only parser rather than importing packaging.

All three reproduced as lab cases in the negative battery before fixing (39 checks, all pass); full pre-commit run --all-files green.

haonan3 added 2 commits July 31, 2026 21:22
- refl seed: replace the null-tolerance with a loud ValueError — roles.py
  documents fixed-noise as the verified DRaFT regime and no live config
  produces seed:null; silently unseeding violated the package's own
  contract and diverged from the mainline base classes
- closed-PR label cleanup: delete all four labels unconditionally (the
  payload snapshot misses a label written by a review-sync racing the
  close instant, and nothing ever repairs a closed PR); tolerate only
  HTTP 404 on deletes here and in the review consumer — blanket || true
  would turn permission regressions (the one failure class observed in
  production) into green runs
- review-labels concurrency: fork PRs fall back to a head-repo+branch
  group (actual serialization) instead of per-run; honest comments
- boundaries checker: relative-import level bound now matches Python
  (level <= len(pkg)); SKIP_PARTS intersects repo-relative parts so a
  checkout living under a directory named vendor can't blank the rules;
  follower set gains tab (valid PEP 508 whitespace)
- bagel guard comment documents why bundle.transformer suffices (vae/vit
  stay in load-time eval and are never mode-flipped)
- gitignore: converter-output jsonl patterns cover nested dirs (**/)
@haonan3
haonan3 merged commit 683a59b into main Jul 31, 2026
7 checks passed
@github-actions github-actions Bot removed the need review Ready and waiting for review label Jul 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant