Skip to content

test(runtime): cover Windows external artifact publication - #461

Merged
zeegin merged 7 commits into
mainfrom
codex/issue-310-windows-publication-contract
Aug 12, 2026
Merged

test(runtime): cover Windows external artifact publication#461
zeegin merged 7 commits into
mainfrom
codex/issue-310-windows-publication-contract

Conversation

@zeegin

@zeegin zeegin commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Closes #310.
Related: #264.

Summary

  • add a win-x64 packaged-tool contract that runs the real v8-runner make flow with a compiled Rust Designer stub;
  • cover both publication into a missing relative output and atomic replacement of an existing output;
  • validate the JSON envelope, published EPF bytes, package artifacts, replacement semantics, and cleanup of .artifacts-stage-* / backup metadata;
  • document the design, binary provenance, and the witnessed RED/GREEN procedure.

Root cause

Unica 0.11.0 bundled v8-runner source commit 72d346c0a8fcf8373d9388257d11e6bef0ad70b2. On Windows it opens the publication directory for fsync using Unix semantics after the Designer step has succeeded, so final directory publication exits 3 with os error 3 or os error 5.

The production fix already arrived through alkoleft/v8-runner-rust#48 and Unica's locked v8-runner-nightly-master-build.2 refresh. This PR adds the missing Windows consumer regression so the packaged bytes must keep that behavior.

RED / GREEN evidence

  • RED — Unica 0.11.0 runner (72d346c0a8fcf8373d9388257d11e6bef0ad70b2) invoked the platform stub and exited 3 with failed to publish staged external directory ... (os error 3).
  • GREEN — current locked runner (7ce1b062843d86644fe55741dbe0ee79f7ca767d) from v8-runner-nightly-master-build.2, SHA-256 191a3d7c930007377238dda0543d1e42cc1a1bd4b209736d54fd41c0ffaac32e, returned [] after both new-target and replacement publication.

Validation

  • 4 focused Python contract tests pass.
  • python -m py_compile scripts/ci/check-tool-contracts.py tests/ci/test_product_contracts.py
  • python tests/ci/test_architecture_registry.py — 40 passed.
  • python scripts/ci/check-rust-platform-boundary.py
  • python scripts/ci/check-architecture-sync.py --base origin/main
  • git diff --check origin/main...HEAD

Local baseline notes:

  • test_product_contracts.py: 39 pass; the pre-existing Windows /missing/v8-runner slash assertion also fails on unmodified origin/main.
  • test_design_documents.py: 8 pass; the pre-existing first-header assertion reports six older base documents, while the new document opens with Date / Status / Decision.
  • full tests/ci discovery reached the five-minute local limit inside an unrelated cargo run; required GitHub CI remains authoritative.

Scope note

The locked upstream runner currently also moves an unreported Alpha.epf.meta.json sidecar into the published output. That pre-existing cross-platform behavior is not the Windows fsync defect from #310. This PR requires exactly the expected EPF package and rejects temporary .artifacts-* residue without making that sidecar part of Unica's required contract.

Summary by CodeRabbit

  • New Features

    • Added contract validation for Windows external artifact publication.
    • Publication checks now verify artifact contents, metadata, replacement behavior, rollback/backup scenarios, and temporary-file cleanup.
    • Added targeted Windows smoke coverage for successful and failed publication cases.
  • Documentation

    • Documented the Windows publication contract, compatibility expectations, error handling, verification scenarios, and test coverage.

@coderabbitai

coderabbitai Bot commented Aug 12, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: c484953a-8d9c-41d6-a195-b50d53957c5b

📥 Commits

Reviewing files that changed from the base of the PR and between cea9fca and 1bf8b2e.

📒 Files selected for processing (4)
  • docs/design/2026-08-12-windows-external-artifact-publication-contract-design.md
  • docs/plans/2026-08-12-windows-external-artifact-publication-contract.md
  • scripts/ci/check-tool-contracts.py
  • tests/ci/test_product_contracts.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • docs/design/2026-08-12-windows-external-artifact-publication-contract-design.md

📝 Walkthrough

Walkthrough

Adds a Windows-only v8-runner contract check for external EPF publication. It runs the packaged runner with a Rust platform stub, validates JSON and filesystem results, checks replacement and cleanup behavior, wires targeted execution, and adds validator and routing tests.

Changes

Windows external publication contract

Layer / File(s) Summary
Contract definition and verification plan
docs/design/..., docs/plans/...
Documents the Windows publication contract, runner comparison steps, error handling, skip rules, validation requirements, and implementation tasks.
Validator and runner integration
scripts/ci/check-tool-contracts.py
Adds bounded Rust-stub compilation, validates publication metadata, EPF bytes, replacement behavior, and cleanup, then registers the contract.
Validator and routing tests
tests/ci/test_product_contracts.py
Covers compilation timeouts, clean and rejected publication results, and win-x64 routing with error propagation.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ContractCheck
  participant V8Runner
  participant RustPlatformStub
  participant OutputDirectory
  ContractCheck->>V8Runner: Run make with external-processors
  V8Runner->>RustPlatformStub: Invoke platform stub
  RustPlatformStub-->>V8Runner: Return platform response
  V8Runner->>OutputDirectory: Publish EPF and metadata
  ContractCheck->>OutputDirectory: Validate bytes and cleanup state
Loading

Possibly related PRs

Suggested reviewers: agrajaga

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: adding runtime coverage for Windows external artifact publication.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/issue-310-windows-publication-contract

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@docs/design/2026-08-12-windows-external-artifact-publication-contract-design.md`:
- Around line 14-15: Keep the `#310` issue reference on the same paragraph line as
the preceding sentence, removing the line break before it; preserve the
surrounding rollback and reproduction references unchanged.

In `@scripts/ci/check-tool-contracts.py`:
- Around line 621-628: The three Rust compilation subprocesses in
check-tool-contracts.py must be bounded and handle timeouts consistently. Add a
regression test in test_product_contracts.py that mocks subprocess.run raising
TimeoutExpired, then update each rustc call to use a 60-second timeout and
catch/report a clearly labeled compilation-timeout error while preserving
existing failure handling.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: a6782cbe-2f06-47fb-b314-d29747913553

📥 Commits

Reviewing files that changed from the base of the PR and between 237fbec and cea9fca.

📒 Files selected for processing (4)
  • docs/design/2026-08-12-windows-external-artifact-publication-contract-design.md
  • docs/plans/2026-08-12-windows-external-artifact-publication-contract.md
  • scripts/ci/check-tool-contracts.py
  • tests/ci/test_product_contracts.py

Comment thread docs/design/2026-08-12-windows-external-artifact-publication-contract-design.md Outdated
Comment thread scripts/ci/check-tool-contracts.py Outdated
@zeegin
zeegin merged commit 2536f9f into main Aug 12, 2026
21 checks passed
@korolevpavel

Copy link
Copy Markdown
Contributor

Независимое post-merge review exact HEAD f9258109f00244c37c0dac884d46aa46d55d158f.

Windows job и новый contract smoke на checksum-verified packaged v8-runner.exe прошли. При этом остаются два воспроизводимых P2 и один P3; поскольку PR уже слит, их следует исправлять отдельным follow-up от актуального main.

  1. P2 — contract допускает false pass без контрольной выгрузки собранного EPF.

    scripts/ci/check-tool-contracts.py:619-635 записывает один общий marker при любом запуске stub, а :717-731 проверяет только наличие этого marker. Поэтому проверка не доказывает, что runner вызвал оба обязательных шага: /LoadExternalDataProcessorOrReportFromFiles и /DumpExternalDataProcessorOrReportToFiles.

    Я независимо повторил adversarial case: fake runner вызывает только /Load..., ни разу не вызывает /Dump..., удаляет старый Deploy и переименовывает stage напрямую, после чего возвращает ожидаемый JSON. Текущий helper вернул [].

    Это расходится с design («полный make», проверка артефакта до публикации) и ожидаемым поведением [Windows][v8-runner] make для EXTERNAL_DATA_PROCESSORS завершается с кодом 3 при fsync каталога после успешной сборки EPF #264. Нужен сначала RED-тест, затем раздельный журнал/счётчики load и dump с проверкой ожидаемой последовательности после каждого make. Утверждение про atomic replacement тоже сейчас наблюдает лишь конечную замену; либо добавить fault injection и проверить rollback boundary, либо сузить формулировку контракта до replacement semantics.

  2. P2 — timeout не ограничивает дерево процессов.

    Новый запуск runner в scripts/ci/check-tool-contracts.py:699-715 и общий compile helper используют subprocess.run(timeout=...). Python завершает непосредственный процесс, но не гарантирует завершение его descendants. В воспроизведении fake runner породил дочерний процесс и завис: helper вернул timeout, а descendant после этого оставался жив (descendant_alive ... True; затем я его принудительно очистил).

    На Windows такой процесс может продолжать удерживать .exe и временные каталоги, из-за чего bounded failure превращается в утечку или сбой TemporaryDirectory cleanup. Нужна containment-модель дерева процессов (Windows Job Object с KILL_ON_JOB_CLOSE; на Unix отдельная process group), завершение и ожидание всего дерева в finally, плюс regression с grandchild, доказывающий его смерть и очистку fixture.

  3. P3 — positive fixture содержит поля, которые validator не проверяет.

    scripts/ci/check-tool-contracts.py:372-411 принимает неверные artifacts.root_dir, items[].kind и execution.payload.output_path. Crafted envelope с root_dir="Outside", kind="platform_log" и payload output_path="Outside" вернул []. Если поля входят в contract — нужны проверки и mutation-тесты; если нет — их лучше убрать из positive fixture, чтобы не создавать ложного покрытия.

Проверено дополнительно:

  • CI routing запускает этот contract на windows-latest при изменении lock/package contour;
  • используется именно packaged binary, связанный с tools.lock.json;
  • EPF и ERF проходят общий upstream publication path;
  • test_product_contracts.py — 41/41, test_design_documents.py — 9/9;
  • py_compile, platform boundary, architecture sync и git diff --check прошли;
  • все обязательные GitHub checks на merge HEAD зелёные.

@zeegin zeegin added this to the v0.12 milestone Aug 12, 2026
@zeegin
zeegin deleted the codex/issue-310-windows-publication-contract branch August 16, 2026 21:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

runtime.make: публикация внешнего артефакта падает на Windows при fsync каталога

2 participants