diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index bf2cf97..ffca5d4 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -112,7 +112,7 @@ Important staging note: `build` and `dump` use cases dispatch by `builder`: - `builder=DESIGNER` uses the existing `DesignerDsl`. -- `builder=IBCMD` uses `IbcmdDsl` with `config import/apply` for build and `config export` for dump; for EDT build the EDT export step still produces Designer-format files first, and for EDT dump the reverse path first updates an internal Designer snapshot before EDT import/publication. +- `builder=IBCMD` uses `IbcmdDsl` with `config import/apply` for build and `config export` for dump; for EDT build the EDT export step still produces Designer-format files first, while EDT dump uses private per-IB Designer and configured-source shadows before merge/publication. - Builder backends are expected to stay interchangeable for implemented builder scenarios. Functionality added for the Designer builder should also be available through the IBCMD builder, or the gap must be documented explicitly. Future Designer agent mode should be added behind the same use-case contract. - Server infobase support is a target contract for all tools; file-only behavior must be documented as a current gap rather than treated as the permanent architecture. @@ -128,10 +128,8 @@ Constraints to keep in mind: ## Dump And Artifact Publication -Full replacement outputs are published through a staging/backup contract governed by [ADR-0015](spec/decisions/0015-atomarnaya-publikatsiya-dump-artifacts-cherez-staging-backup.md). -Full dump writes to a sibling staging directory before replacing the resolved target directory. -Package artifacts write to a sibling staging file before replacing the output file, and external EPF/ERF publication stages the whole output directory before replacing it. -Incremental and partial dump modes remain direct non-atomic update modes. +Package and external EPF/ERF artifacts use the sibling staging/backup contract from [ADR-0015](spec/decisions/0015-atomarnaya-publikatsiya-dump-artifacts-cherez-staging-backup.md). +All dump modes instead execute in private per-IB shadows and publish only managed files through the recoverable manifest transaction from [ADR-0023](spec/decisions/0023-izolirovat-runtime-state-po-infobase-i-ispolzovat-private-shadow.md). A B/S/D conflict publishes nothing; incremental and partial requests are promoted to one full shadow operation when their private baseline is unavailable. ## Output Flow @@ -153,9 +151,9 @@ Use cases now return transport-neutral payloads or structured failures. - `workPath/convert/out///` stores default generated convert outputs; `convert --output ` publishes the same converted source-set content under a caller-provided root using source-set path mirror layout. - `workPath/temp/partial-lists/` stores partial load and partial dump list files. - `workPath/temp/yaxunit/` stores temporary YaXUnit config files. -- `workPath/hash-storages/` remains reserved for change detection state. +- `workPath/ib-state/v1//-/` stores opaque per-IB change-detection state, private CDFI, baselines and recoverable transactions. Legacy `workPath/hash-storages/` is not migrated or reused. - `workPath/designer//` is used by the EDT export/build flow as the generated Designer-format output area for a source-set. -The `source-set` and `workPath` state boundary is formalized in [ADR-0002](spec/decisions/0002-izolirovat-runtime-state-po-source-set-pod-workpath.md): `DESIGNER` format uses one `designer-` change-detection context, while `EDT` format uses both `edt-` for export decisions and `designer-` for load decisions. +The source/runtime boundary is formalized in [ADR-0023](spec/decisions/0023-izolirovat-runtime-state-po-infobase-i-ispolzovat-private-shadow.md): every configured or generated view has a versioned identity scoped by infobase, source root, purpose, format, backend and logical role. EDT build still has distinct configured and generated Designer contexts, but dump platform/configured shadows live only in private transactions. Exclusive command ownership of `workPath` is governed by [ADR-0011](spec/decisions/0011-eksklyuzivnoe-vladenie-workpath-na-vremya-komandy.md). On-demand change detection and conservative file-level partial load rules are governed by [ADR-0012](spec/decisions/0012-on-demand-change-detection-i-faylovaya-partial-load-strategiya.md). diff --git a/Cargo.lock b/Cargo.lock index be388f8..be7f6ef 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -2383,6 +2383,7 @@ dependencies = [ "tracing-subscriber", "uuid", "walkdir", + "windows-sys 0.61.2", "zip", ] diff --git a/Cargo.toml b/Cargo.toml index f08b650..bf6f1fd 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -36,6 +36,14 @@ reqwest = { version = "0.12", default-features = false, features = ["blocking", [target.'cfg(windows)'.dependencies] process-wrap = { version = "9.1.0", default-features = false, features = ["std", "job-object", "creation-flags"] } +windows-sys = { version = "0.61", features = [ + "Wdk_Foundation", + "Wdk_Storage_FileSystem", + "Win32_Foundation", + "Win32_Security", + "Win32_Storage_FileSystem", + "Win32_System_IO", +] } [dev-dependencies] assert_cmd = "2" diff --git a/SKILL/SKILL.md b/SKILL/SKILL.md index 2b0e4c5..46649bf 100644 --- a/SKILL/SKILL.md +++ b/SKILL/SKILL.md @@ -32,6 +32,16 @@ v8-runner --json-message build Use text output for direct human diagnostics. +For JSON build and dump results, use each step's `receipt` as the exact source-sync audit record. +`requested` is the byte-exact add/modify/delete delta; `processed`, `skipped`, and `conflicted` +are independent audit dimensions, not a strict partition. In an applied result, the same target +may appear in both `processed` and `skipped` when it belongs to the effective platform scope but +publication retained or no-op'd the local version. Full operations report the complete managed +final scope; incremental/partial operations report observable shadow writes. Inconsistent overlap is invalid. `preHash`/`postHash` are SHA-256 +values, with `null` denoting the absent side of an add/delete. Failed or cancelled steps keep the +computed `requested` delta and leave outcome lists empty; if failure happens before a delta can be +computed, every list may be empty. + Use `v8-runner version` or `v8-runner --version` to check the installed application version; it does not require `v8project.yaml`. Useful global flags: diff --git a/SKILL/references/file-and-artifact-workflows.md b/SKILL/references/file-and-artifact-workflows.md index cf328d1..f829fb0 100644 --- a/SKILL/references/file-and-artifact-workflows.md +++ b/SKILL/references/file-and-artifact-workflows.md @@ -38,7 +38,19 @@ control characters are rejected. With `builder=DESIGNER`, Designer validates whe metadata root type exists. With `builder=IBCMD`, the object list is not used because partial degrades to incremental. -For `format=EDT`, dump uses an internal Designer snapshot under `workPath/designer/`, then imports the result into the EDT target. +For `format=EDT`, dump uses private platform and configured-source shadows inside the scoped +`ib-state/v1` transaction. The platform never writes the project source or +`workPath/designer/` as its dump target. + +Dump safety and recovery rules: + +- each infobase/source-set identity owns private shadows and generation-scoped baselines under `workPath/ib-state/v1`; fingerprints are opaque and state must never be reused across infobases; +- legacy `workPath/hash-storages` is deliberately not migrated; missing scoped state means full bootstrap, never an unchanged result; +- incremental and partial dump require a valid matching baseline and private `ConfigDumpInfo.xml`; missing or corrupt state promotes that one operation to a full dump; +- receipt lists are exact but independent audit dimensions: an applied target may be both `processed` and `skipped` when platform work occurred but publication retained/no-op'd it; +- a three-way conflict publishes no project-source files or new private generation; +- `B=absent, S=present, D=absent` is a conflict; runner never deletes a local file absent from its baseline; +- full, incremental, and partial modes all use recoverable manifest publication. Forward recovery requires the exact `(generation, UUID transaction token)`; otherwise it restores the previous managed file state. ## Convert @@ -94,4 +106,4 @@ Behavior: - external data processors and reports publish `.epf` / `.erf` into the output directory; - `builder=DESIGNER` is required. -Full dump and package/external artifact publication use staged publication with backup/rollback semantics. Incremental and partial dump are non-atomic update modes. +Package and external-artifact publication uses staged backup/rollback semantics. Every dump mode uses journaled, manifest-scoped source publication and private-state recovery. diff --git a/SKILL/references/troubleshooting.md b/SKILL/references/troubleshooting.md index ebf4457..a0ae4c4 100644 --- a/SKILL/references/troubleshooting.md +++ b/SKILL/references/troubleshooting.md @@ -47,7 +47,7 @@ workPath/temp//runs// Useful `workPath` locations: -- `workPath/hash-storages/`: persisted change-detection state. +- `workPath/ib-state/v1/`: opaque per-infobase/per-source CDFI, baselines, observations, and recovery journals. Do not edit, copy between infobases, or delete it as a routine fix; legacy `hash-storages` is not migrated. - `workPath/edt-workspace/`: shared EDT workspace for `init`. - `workPath/convert/edt-workspace/`: separate EDT workspace for `convert`. - `workPath/designer//`: generated Designer representation, especially for EDT flows. @@ -55,3 +55,7 @@ Useful `workPath` locations: - `workPath/logs/platform/`: platform logs. - `workPath/temp/partial-lists/`: Designer partial load/dump list files; failed partial-load builds preserve the relevant list file for diagnostics. - `workPath/temp/`: temporary run artifacts and diagnostics. + +If dump reports a three-way conflict, inspect `git diff` and resolve the local/source intent before +retrying. Conflict publishes no source files and advances no private generation. For a deliberate +build recovery use `v8-runner build --full-rebuild`; do not repair opaque state files manually. diff --git a/docs/CAPABILITIES.md b/docs/CAPABILITIES.md index 6483c06..007207d 100644 --- a/docs/CAPABILITIES.md +++ b/docs/CAPABILITIES.md @@ -26,14 +26,14 @@ CLI help, доверяйте текущему коду и затем синхр | `tools download ` | CLI-only загрузка latest releases | Загружает выбранный YAxUnit, Vanessa Automation single или onec-client-mcp-devkit; обновляет local overlay для Vanessa/client MCP и при `yaxunit --sources` добавляет YAxUnit как `source-set` `tests` | | `init` | `format=DESIGNER` + `builder=DESIGNER` | Создаёт файловую ИБ через Designer; server connection остаётся manual prerequisite | | `init` | `format=DESIGNER` + `builder=IBCMD` | Выполняет `ensure` файловой или серверной ИБ через `ibcmd infobase create` | -| `init` | `format=EDT` + `builder=DESIGNER|IBCMD` | Готовит ИБ по правилам builder и импортирует EDT workspace | +| `init` | `format=EDT` + `builder=DESIGNER\|IBCMD` | Готовит ИБ по правилам builder и импортирует EDT workspace | | `extensions` | `format=DESIGNER` или `format=EDT` | Обновляет свойства extension `source-set` | -| `build` | `format=DESIGNER` + `builder=DESIGNER|IBCMD` | Выполняет incremental/full загрузку в ИБ | -| `build` | `format=EDT` + `builder=DESIGNER|IBCMD` | Экспортирует изменённые EDT `source-set`, затем грузит generated Designer output | +| `build` | `format=DESIGNER` + `builder=DESIGNER\|IBCMD` | Выполняет incremental/full загрузку в ИБ | +| `build` | `format=EDT` + `builder=DESIGNER\|IBCMD` | Экспортирует изменённые EDT `source-set`, затем грузит generated Designer output | | `test` | Та же матрица, что и у `build` | Всегда сначала запускает `build` | | `dump` | `format=DESIGNER` + `builder=DESIGNER` | Полная, инкрементальная или object-scoped partial выгрузка | | `dump` | `format=DESIGNER` + `builder=IBCMD` | Полная и инкрементальная выгрузка; `partial` деградирует в incremental с warning; standalone-server state изолирован в `workPath/ibcmd-data` | -| `dump` | `format=EDT` + `builder=DESIGNER|IBCMD` | Reverse sync из ИБ через internal Designer snapshot и EDT import | +| `dump` | `format=EDT` + `builder=DESIGNER\|IBCMD` | Reverse sync через private Designer/configured-source shadows и EDT import | | `convert` | CLI-only repo-aware конвертация текущих `source-set` | Не использует `builder` и не требует ИБ | | `load` | `format=DESIGNER` + `builder=DESIGNER` | Загрузка `.cf` / `.cfe` артефактов в ИБ | | `make` / `artifacts` | `format=DESIGNER` + `builder=DESIGNER` | Экспорт `.cf` / `.cfe` и публикация `.epf` / `.erf` | @@ -176,12 +176,17 @@ v8-runner build [--source-set ] [--full-rebuild] - Для `DESIGNER` выбирает incremental, partial или full path по изменённым файлам выбранного scope. - Для `EDT` сначала анализирует и экспортирует выбранные EDT `source-set`, затем грузит generated Designer files выбранным backend. +- Designer backend получает private source transaction, а не project source tree. Source + `ConfigDumpInfo.xml` игнорируется; private CDFI/baseline/observation публикуются одной generation + только после successful pipeline. +- Отсутствие scoped `ib-state/v1` или missing/corrupt private CDFI означает full bootstrap; + legacy `workPath/hash-storages` не мигрируется. - После успешного project stage, включая scoped `--source-set`, подготавливает `tools.client_mcp.extension`, если оно настроено: `source` загружается как extension из исходников, `.cfe` `artifact` загружается как extension с именем `tools.client_mcp.extension.name`. -- Для source-backed `tools.client_mcp.extension` использует отдельное состояние change detection - под `workPath/hash-storages`: неизменённый source пропускает export/load, `--full-rebuild` +- Для source-backed `tools.client_mcp.extension` использует отдельное per-IB/per-source состояние + под `workPath/ib-state/v1`: неизменённый source пропускает export/load, `--full-rebuild` принудительно обновляет расширение. - `tools.client_mcp.extension` не является project `source-set`; `--source-set` выбирает только project source-set. @@ -254,8 +259,14 @@ v8-runner dump --mode [--source-set ] [--extens потому что деградирует в incremental. - `builder=DESIGNER` поддерживает true object-scoped partial. - `builder=IBCMD` не умеет object-scoped partial; запрос деградирует в incremental с warning. -- `format=EDT` использует internal Designer snapshot под `workPath/designer/`, - затем импортирует его в EDT target и публикует результат атомарной заменой target каталога. +- Все режимы выполняются в private shadow. Missing baseline/private CDFI повышает + incremental/partial request до full shadow dump. +- B/S/D conflict, включая `B=absent, S=present, D=absent`, или TOCTOU mismatch публикует ноль + source files и не продвигает private generation. +- Conflict-free managed files публикуются через recoverable journal; exact generation и UUID + transaction token не позволяют принять чужую одноимённую generation за завершённый dump. +- Для `format=EDT` private Designer shadow импортируется в private EDT/configured shadow до той же + manifest publication; platform не пишет в project source или `workPath/designer/`. ### `convert` @@ -376,7 +387,7 @@ v8-runner mcp serve http Важные runtime директории: -- `workPath/hash-storages/`: persisted change-detection state. +- `workPath/ib-state/v1/`: opaque per-IB/per-source CDFI, baselines, observations and transaction journals; не редактируйте и не копируйте state между ИБ. - `workPath/edt-workspace/`: общий EDT workspace для `init`. - `workPath/convert/edt-workspace/`: отдельный EDT workspace для `convert`. - `workPath/ibcmd-data/`: изолированный standalone-server data directory для IBCMD dump; это runtime state `v8-runner`, его можно удалить, когда нет активных CLI/MCP команд проекта. diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index a6bf1ec..539ed95 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -257,13 +257,17 @@ tests: Корень runtime state: -- `workPath/hash-storages` +- `workPath/ib-state/v1//-` - `workPath/logs` - `workPath/temp` - `workPath/edt-workspace` - `workPath/designer` Если каталога нет, он создаётся автоматически. +Fingerprints являются opaque и зависят от secret-free identity ИБ и source context; их не +следует вычислять или редактировать вручную. Legacy `workPath/hash-storages` не мигрируется +и не переиспользуется. Первое обращение к scoped state выполняет full bootstrap, а не +интерпретирует отсутствующий snapshot как отсутствие изменений. ### `execution_timeout` @@ -476,9 +480,9 @@ MCP endpoint и не гарантирует наличие Vanessa tools. `tools.client_mcp.extension` не добавляется в `source-set` и не выбирается через `--source-set`. `init` импортирует EDT `source` в workspace, `build` подготавливает расширение после project source-set build, а `launch mcp` и `launch mcp va` расширение не устанавливают и не обновляют. -Для `source` build хранит отдельный snapshot под `workPath/hash-storages`: повторный запуск с -неизменёнными исходниками пропускает export/load, а `build --full-rebuild` принудительно -обновляет расширение. +Для `source` build хранит отдельный per-IB/per-source snapshot под `workPath/ib-state/v1`: +повторный запуск с неизменёнными исходниками пропускает export/load, а первое обращение или +`build --full-rebuild` выполняет full operation. Legacy snapshot не переиспользуется. `v8-runner tools download client-mcp` может заполнить этот блок в `v8project.local.yaml`: с `--sources` он указывает `source.path` на diff --git a/docs/DEEP_DIVE.md b/docs/DEEP_DIVE.md index 9295445..cb447ca 100644 --- a/docs/DEEP_DIVE.md +++ b/docs/DEEP_DIVE.md @@ -31,16 +31,18 @@ MCP DTO в одном слое. `source-set` — минимальная единица оркестрации. -- Для `format=DESIGNER` используется один runtime context `designer-`. +- Для `format=DESIGNER` используется один логический Designer context на source-set. - Для `format=EDT` используются два context-а: - - `edt-` для решения, нужен ли export; - - `designer-` для решения, что именно грузить в ИБ. -- Persisted state живёт в `workPath/hash-storages/`. + - EDT-source context для решения, нужен ли export; + - generated-Designer context для решения, что именно грузить в ИБ. +- Каждый context изолирован по ИБ и source identity под `workPath/ib-state/v1`. - Generated Designer output для EDT flow живёт под `workPath/designer/`. Change detection выполняется on-demand во время build/export/load decision и не требует background watcher. `build --source-set ` ограничивает анализ, export/load decision и runtime snapshot commit только указанным source-set. +Legacy `workPath/hash-storages` намеренно не мигрируется: первое обращение к scoped state, +включая пустой source-set, является full bootstrap, а не `NoChanges`. ## Пайплайн `build` @@ -48,8 +50,10 @@ runtime snapshot commit только указанным source-set. 1. Анализ изменений по выбранным `source-set`. 2. Выбор partial/full path по изменённым файлам. -3. Загрузка через выбранный backend. -4. Commit runtime snapshot только после успешного шага. +3. Копирование managed source в private transaction; source `ConfigDumpInfo.xml`, symlinks и + вложенный `workPath` исключаются, валидный private CDFI seed добавляется отдельно. +4. Загрузка через выбранный backend только из private staging. +5. Commit CDFI, baseline и source observation одной recoverable generation после успеха. Для `EDT`: @@ -60,6 +64,8 @@ runtime snapshot commit только указанным source-set. Пайплайн намеренно не является атомарным across many `source-set`: поздний failure не откатывает уже успешные ранние шаги. +Missing/corrupt private CDFI приводит к full bootstrap. Platform не получает живое source tree, +поэтому failed/cancelled build не создаёт и не изменяет source `ConfigDumpInfo.xml`. ## Проверка и тесты @@ -82,7 +88,19 @@ runtime snapshot commit только указанным source-set. - Для `DESIGNER` может быть full, incremental или partial. - Для `IBCMD` object-scoped partial деградирует в incremental. -- Для `format=EDT` использует internal Designer snapshot, затем EDT import. +- Любой режим выполняется в private complete shadow; отсутствие matching baseline/private CDFI + повышает incremental/partial request до одной full shadow operation. +- Публикация сравнивает baseline `B`, текущий source `S` и dump `D`; любой conflict или TOCTOU + mismatch оставляет весь source-set и private generation неизменными. +- `B=absent, S=present, D=absent` является conflict, а не разрешением удалить локальный файл. +- Managed-file journal обеспечивает restart recovery. Forward recovery требует точной пары + `(generation, UUID transaction token)`; совпавшая generation другой операции не считается успехом. +- Для `format=EDT` private platform shadow импортируется в private configured-source shadow до + той же B/S/D publication; platform не пишет в project source или `workPath/designer/`. +- Exact receipt lists — независимые audit dimensions: одинаковый target может быть одновременно + `processed` и `skipped`, если он входил в effective platform scope, но merge сохранил/no-op + локальный файл. Full сообщает полный managed result, incremental/partial — наблюдаемые записи + private shadow. ### `convert` diff --git a/docs/plans/2026-07-21-issue-30-per-ib-runtime-state.md b/docs/plans/2026-07-21-issue-30-per-ib-runtime-state.md new file mode 100644 index 0000000..ba53b50 --- /dev/null +++ b/docs/plans/2026-07-21-issue-30-per-ib-runtime-state.md @@ -0,0 +1,142 @@ +# Issue #30: per-IB runtime state and private shadow implementation plan + +> Execution discipline: use test-driven development for every behavior change, +> independent tester/reviewer/Rust-expert passes before commits, and explicit waivers +> for findings that cannot be fixed in this PR. + +**Goal:** Prevent false incremental decisions across infobases and guarantee that +build/dump never exposes platform-owned CDFI or half-applied changes in source trees. + +**Architecture:** Introduce a secret-free `RuntimeStateLayout` at the domain/use-case +boundary and make change detection consume resolved storage paths. Designer build runs +against a private source copy. Dump runs against a private complete shadow and publishes +through a pure three-way merge plus an owned-file journal transaction. Receipts are +shared serializable domain values populated from hash-bearing deltas. + +**Tech stack:** Rust 2021, `sha2`, `redb`, `walkdir`, existing platform fakes and CLI +integration tests. + +--- + +## Task 1: Runtime identity and versioned layout + +**Files:** + +- Create: `src/domain/runtime_state.rs` +- Modify: `src/domain/mod.rs`, `src/domain/source_set.rs` +- Modify: `src/change_detection/source_sets.rs`, `src/change_detection/analyzer.rs` +- Test: module tests in the files above + +1. Write failing tests for equivalent plain/raw connection forms, nonexistent and + symlinked paths, embedded auth stripping, secret-safe `Debug`, distinct IBs, + distinct roots/formats/backends, and the exact versioned layout. +2. Implement one fallible typed connection/path normalizer, tagged length-prefixed + SHA-256 identity and `RuntimeStateLayout`; reject unsupported raw forms. +3. Resolve contexts with explicit state directories and remove the legacy + `workPath/hash-storages` derivation. +4. Represent absent state as `Bootstrap`, never `NoChanges`; keep `IbBaseline` and + `SourceObservation` as distinct types tied to one state generation. +5. Run targeted tests, format and Clippy. + +## Task 2: Shared source inventory and exact hash deltas + +**Files:** + +- Modify: `src/change_detection/scanner.rs`, `src/change_detection/analyzer.rs` +- Create: `src/domain/sync_receipt.rs` +- Modify: `src/domain/mod.rs`, `src/domain/build.rs`, `src/domain/dump.rs` +- Test: scanner/analyzer/domain tests + +1. Write failing tests for case-aware CDFI exclusion, symlink exclusion, custom nested + `workPath` exclusion and pre/post hashes for add/modify/delete. +2. Add explicit excluded roots and one reusable source-file predicate. +3. Enrich `FileChange` and prepared state with stable relative paths and hashes. +4. Add sorted `SyncReceipt` / `SyncTarget` serialization with private fields and + exhaustive terminal smart constructors; attach one receipt to every `BuildStep` and + one to `DumpResult`. +5. Test add/delete null hash semantics, unchanged partial closure, failed/cancelled + requested-only receipts and rejection of contradictory terminal states. +6. Run targeted tests and schema/snapshot checks. + +## Task 3: Private Designer build transaction + +**Files:** + +- Create: `src/use_cases/runtime_state.rs`, `src/use_cases/source_transaction.rs` +- Modify: `src/use_cases/mod.rs`, `src/use_cases/build_project.rs` +- Modify: `src/use_cases/build_project/coordinator.rs`, helpers as required +- Test: build-project unit and CLI fake-platform tests + +1. Write failing tests proving source CDFI is ignored, source remains byte-identical, + private CDFI is seeded only when structurally valid, and missing/corrupt CDFI forces full. +2. Copy source to an owned transaction directory without following symlinks or copying + excluded roots/files; compute partial paths from original root. +3. Point Designer load at staging and stage the produced validated CDFI after successful + apply for the same recoverable generation commit as baseline/hash state. +4. Do not auto-retry an ambiguous platform rejection of seeded CDFI; preserve prior + state and require explicit full rebuild. Make CDFI and source hashes visible through + one recoverable generation; ensure all failure/cancellation paths + leave source and prior state unchanged. +5. Apply the same runtime layout to tool-extension Designer flows. + +## Task 4: Per-IB state through EDT and IBCMD + +**Files:** + +- Modify: `src/use_cases/build_project.rs`, coordinator/helpers +- Modify: `src/use_cases/tool_extension.rs` +- Test: use-case tests + +1. Write lifecycle tests: A build/full, A repeat/skip, B build/full, A/skip, restart/skip. +2. Thread resolved per-IB contexts through Designer, IBCMD, EDT export and generated + Designer load stages. +3. Defer EDT snapshot commit until its downstream Designer load/apply succeeds. +4. Verify old storage is untouched and never read. + +## Task 5: Private dump shadow and pure three-way merge + +**Files:** + +- Create: `src/use_cases/shadow_merge.rs` +- Modify: `src/use_cases/dump_config.rs`, coordinator/helpers +- Modify: `src/use_cases/staged_publication.rs` if source-set transaction support is needed +- Test: merge unit tests and dump fake-platform integration tests + +1. Write table-driven tests for every `B/S/D` rule, absent baseline bootstrap, delete/add, + CDFI exclusion, TOCTOU rejection, crash rollback and preservation of symlink/nested + workPath/ignored entries. +2. Maintain a typed complete private `IbBaseline` and execute every dump mode into a + private shadow; upgrade first incremental/partial request to full shadow dump. +3. Build an owned-file manifest transaction with byte-exact backups and a durable journal; + publish nothing on any conflict or revalidation mismatch, and recover unfinished + publication before the next analysis. +4. After source publication, commit baseline/CDFI/`SourceObservation` under one recoverable + state generation. Advance applied/converged paths; retain old state for local-only paths; + advance nothing on conflict. +5. Populate exact requested/processed/skipped/conflicted dump receipts. + +## Task 6: Public diagnostics and documentation + +**Files:** + +- Modify: CLI/MCP output tests or snapshots only where additive receipts surface +- Modify: `spec/decisions/README.md`, ADR-0002, ADR-0012, ADR-0015 +- Modify: `spec/architecture/invariants.md`, affected arc42 views +- Modify: `docs/CONFIGURATION.md`, `docs/DEEP_DIVE.md`, `SKILL/SKILL.md` + +1. Add tests for deterministic receipt JSON and sanitized diagnostics. +2. Document `ib-state/v1`, deliberate non-migration/full bootstrap and conflict recovery. +3. Update the repo-local skill with short external guidance only. + +## Task 7: Verification and pull request + +1. Run targeted tests after every task. +2. Run `cargo fmt --check`, `cargo clippy --all-targets --all-features -- -D warnings`, + `cargo test --all-targets --all-features`, architecture guardrails and repository CI script. +3. Run independent Tester, Reviewer and Rust-expert reviews; fix or record every finding. +4. Run real disposable file-IB acceptance. If a platform installation is unavailable, + preserve an executable acceptance script and open the PR as incomplete validation, + without claiming that #30 is closed. +5. Commit with repository format, push `fix/per-ib-runtime-state`, open an upstream PR + linking #30 and noting that it supersedes the narrower #24 approach. Use `Closes #30` + only when the real Designer acceptance criterion has been recorded. diff --git a/scripts/test/README.md b/scripts/test/README.md index f9f398d..cf13689 100644 --- a/scripts/test/README.md +++ b/scripts/test/README.md @@ -89,7 +89,9 @@ live-mcp-http.py - Это главный исполнитель реального fixture-based smoke. - Он владеет порядком шагов, валидацией config, подготовкой workspace и проверкой результата. -- Для `builder=DESIGNER` выполняет `syntax`, opt-in `test`, упаковку `.cf/.cfe/.epf/.erf` и проверку deploy-ready артефактов. +- Для `builder=DESIGNER` выполняет `syntax`, opt-in `test`, упаковку `.cf/.cfe/.epf/.erf`, + проверку deploy-ready артефактов и обязательный `full/incremental/partial` no-clobber dump + через private shadow без публикации `ConfigDumpInfo.xml` в source. - Для `builder=IBCMD` выполняет `dump full/incremental/partial` smoke вместо designer-specific packaging. - Именно этот скрипт является общим contract layer для `live-cli-designer.sh` и `live-cli-ibcmd.sh`. diff --git a/scripts/test/ci-designer-config.sh b/scripts/test/ci-designer-config.sh index 3f00255..4e45ce3 100755 --- a/scripts/test/ci-designer-config.sh +++ b/scripts/test/ci-designer-config.sh @@ -94,7 +94,7 @@ def inject_platform_path(match: re.Match[str]) -> str: text, platform_count = re.subn( - r"(?m)(?P^tools:\n platform:\n)(?P(?: .*\n)+)", + r"(?m)(?P^ platform:\n)(?P(?: .*(?:\n|\Z))+)", inject_platform_path, text, count=1, diff --git a/scripts/test/live-cli-fixture.sh b/scripts/test/live-cli-fixture.sh index e199950..c330547 100755 --- a/scripts/test/live-cli-fixture.sh +++ b/scripts/test/live-cli-fixture.sh @@ -275,6 +275,73 @@ if payload.get("command") != expected_command: PY } +assert_json_dump_conflict() { + local json_path="$1" + local expected_mode="$2" + local expected_path="$3" + + python3 - "$json_path" "$expected_mode" "$expected_path" <<'PY' +import json +import sys + +json_path, expected_mode, expected_path = sys.argv[1], sys.argv[2], sys.argv[3] +with open(json_path, "r", encoding="utf-8") as fh: + payload = json.load(fh) + +if payload.get("ok") is not False or payload.get("command") != "dump": + raise SystemExit(f"dump must return a structured failure: {payload}") + +data = payload.get("data", {}) +receipt = data.get("receipt", {}) +if data.get("mode") != expected_mode: + raise SystemExit(f"unexpected dump mode, expected {expected_mode}: {payload}") +if receipt.get("status") != "conflict": + raise SystemExit(f"dump must report a conflict receipt: {payload}") +if receipt.get("processed") != []: + raise SystemExit(f"conflicted dump must process no source targets: {payload}") +if not receipt.get("conflicted"): + raise SystemExit(f"conflicted dump must identify exact targets: {payload}") +if receipt.get("requested") != receipt.get("conflicted"): + raise SystemExit(f"conflict receipt requested/conflicted targets must match: {payload}") +if [target.get("path") for target in receipt.get("conflicted", [])] != [expected_path]: + raise SystemExit(f"dump must identify the exact conflicted target {expected_path}: {payload}") +target = receipt["conflicted"][0] +if not isinstance(target.get("preHash"), str) or target.get("postHash") is not None: + raise SystemExit(f"local-only conflict must contain exact preHash and null postHash: {payload}") +for group in ("requested", "processed", "skipped", "conflicted"): + if any(item.get("path", "").lower().endswith("configdumpinfo.xml") for item in receipt.get(group, [])): + raise SystemExit(f"ConfigDumpInfo.xml must not appear in source receipt group {group}: {payload}") +PY +} + +assert_json_dump_applied() { + local json_path="$1" + local expected_mode="$2" + + python3 - "$json_path" "$expected_mode" <<'PY' +import json +import sys + +json_path, expected_mode = sys.argv[1], sys.argv[2] +with open(json_path, "r", encoding="utf-8") as fh: + payload = json.load(fh) + +data = payload.get("data", {}) +receipt = data.get("receipt", {}) +if payload.get("ok") is not True or data.get("ok") is not True: + raise SystemExit(f"dump must succeed: {payload}") +if data.get("mode") != expected_mode or receipt.get("status") != "applied": + raise SystemExit(f"dump must return an applied {expected_mode} receipt: {payload}") +if receipt.get("conflicted") != []: + raise SystemExit(f"successful dump must contain no conflicts: {payload}") +if not receipt.get("requested") or receipt.get("requested") != receipt.get("processed"): + raise SystemExit(f"full dump must process every requested source target: {payload}") +for group in ("requested", "processed", "skipped", "conflicted"): + if any(item.get("path", "").lower().endswith("configdumpinfo.xml") for item in receipt.get(group, [])): + raise SystemExit(f"ConfigDumpInfo.xml must not appear in source receipt group {group}: {payload}") +PY +} + extract_connection_file_path() { python3 - "$DESIGNER_CONFIG_PATH" <<'PY' import pathlib @@ -430,6 +497,20 @@ run_cli_json_to_file() { "$BIN_PATH" --config "$DESIGNER_CONFIG_PATH" --json-message "$@" | tee "$json_path" } +run_cli_json_expect_conflict() { + local json_path="$1" + shift + echo + echo "==> --json-message $* (expect conflict)" + set +e + "$BIN_PATH" --config "$DESIGNER_CONFIG_PATH" --json-message "$@" | tee "$json_path" + local cli_status="${PIPESTATUS[0]}" + set -e + if [[ "$cli_status" -ne 3 ]]; then + die "Expected dump conflict exit code 3, got $cli_status: $*" + fi +} + run_test_stage() { local json_path="$OUTPUT_ROOT/json/test-stage.json" @@ -470,33 +551,57 @@ run_launch_smoke() { assert_json_command_ok "$launch_json" "launch" } -run_extended_steps() { - local dump_root="$OUTPUT_ROOT/dump" +assert_source_has_no_cdfi() { + if find "$WORK_BASE_PATH" -type f -iname 'ConfigDumpInfo.xml' -print -quit | grep -q .; then + die "ConfigDumpInfo.xml must never be published into the fixture source tree" + fi +} - print_stage "extended dump validation" - rm -f \ - "$WORK_BASE_PATH/$CONFIGURATION_SOURCE_SET_PATH/Configuration.xml" \ - "$WORK_BASE_PATH/$CONFIGURATION_SOURCE_SET_PATH/ConfigDumpInfo.xml" - run_cli dump --mode full --source-set "$CONFIGURATION_SOURCE_SET_NAME" - assert_file_exists "$WORK_BASE_PATH/$CONFIGURATION_SOURCE_SET_PATH/Configuration.xml" - assert_file_exists "$WORK_BASE_PATH/$CONFIGURATION_SOURCE_SET_PATH/ConfigDumpInfo.xml" - snapshot_dir "$WORK_BASE_PATH/$CONFIGURATION_SOURCE_SET_PATH" "$dump_root/full" - - rm -f "$WORK_BASE_PATH/$CONFIGURATION_SOURCE_SET_PATH/ConfigDumpInfo.xml" - run_cli dump --mode incremental --source-set "$CONFIGURATION_SOURCE_SET_NAME" - assert_dir_exists "$WORK_BASE_PATH/$CONFIGURATION_SOURCE_SET_PATH" - assert_file_exists "$WORK_BASE_PATH/$CONFIGURATION_SOURCE_SET_PATH/ConfigDumpInfo.xml" - snapshot_dir "$WORK_BASE_PATH/$CONFIGURATION_SOURCE_SET_PATH" "$dump_root/incremental" - - rm -f "$WORK_BASE_PATH/$CONFIGURATION_SOURCE_SET_PATH/Catalogs/Справочник1.xml" - run_cli dump --mode partial --source-set "$CONFIGURATION_SOURCE_SET_NAME" --object Catalog.Справочник1 - assert_file_exists "$WORK_BASE_PATH/$CONFIGURATION_SOURCE_SET_PATH/Catalogs/Справочник1.xml" - snapshot_dir "$WORK_BASE_PATH/$CONFIGURATION_SOURCE_SET_PATH" "$dump_root/partial" - - rm -f "$WORK_BASE_PATH/$EXTENSION_SOURCE_SET_PATH/ConfigDumpInfo.xml" - run_cli dump --mode incremental --source-set "$EXTENSION_SOURCE_SET_NAME" --extension "$EXTENSION_SOURCE_SET_NAME" - assert_file_exists "$WORK_BASE_PATH/$EXTENSION_SOURCE_SET_PATH/ConfigDumpInfo.xml" - snapshot_dir "$WORK_BASE_PATH/$EXTENSION_SOURCE_SET_PATH" "$dump_root/extension-incremental" +assert_private_cdfi_exists() { + if ! find "$WORK_PATH/ib-state/v1" -type f -iname 'ConfigDumpInfo.xml' -print -quit | grep -q .; then + die "Expected private ConfigDumpInfo.xml under per-infobase runtime state" + fi +} + +run_designer_runtime_state_steps() { + local dump_root="$OUTPUT_ROOT/dump" + local source_root="$WORK_BASE_PATH/$CONFIGURATION_SOURCE_SET_PATH" + local source_before="$dump_root/source-before-full" + local conflict_before="$dump_root/source-before-conflicts" + local state_before="$dump_root/state-before" + + print_stage "private dump shadow and no-clobber validation" + assert_source_has_no_cdfi + assert_private_cdfi_exists + snapshot_dir "$source_root" "$source_before" + + rm -rf "$source_root" + mkdir -p "$source_root" + + local full_json="$OUTPUT_ROOT/json/dump-full-applied.json" + run_cli_json_to_file "$full_json" dump --mode full --source-set "$CONFIGURATION_SOURCE_SET_NAME" + assert_json_dump_applied "$full_json" "FULL" + assert_file_exists "$source_root/Configuration.xml" + assert_source_has_no_cdfi + assert_private_cdfi_exists + + printf 'local-only source change\n' > "$source_root/LocalOnly.txt" + snapshot_dir "$source_root" "$conflict_before" + snapshot_dir "$WORK_PATH/ib-state/v1" "$state_before" + + local incremental_json="$OUTPUT_ROOT/json/dump-incremental-conflict.json" + run_cli_json_expect_conflict "$incremental_json" dump --mode incremental --source-set "$CONFIGURATION_SOURCE_SET_NAME" + assert_json_dump_conflict "$incremental_json" "INCREMENTAL" "LocalOnly.txt" + diff -qr "$conflict_before" "$source_root" + diff -qr "$state_before" "$WORK_PATH/ib-state/v1" + + local partial_json="$OUTPUT_ROOT/json/dump-partial-conflict.json" + run_cli_json_expect_conflict "$partial_json" dump --mode partial --source-set "$CONFIGURATION_SOURCE_SET_NAME" --object Document.Документ1 + assert_json_dump_conflict "$partial_json" "PARTIAL" "LocalOnly.txt" + diff -qr "$conflict_before" "$source_root" + diff -qr "$state_before" "$WORK_PATH/ib-state/v1" + assert_source_has_no_cdfi + assert_private_cdfi_exists } if [[ -z "$DESIGNER_CONFIG_PATH" ]]; then @@ -543,18 +648,14 @@ if ! extract_connection_file_path >/dev/null; then die "Live Designer config must use file-based infobase.connection ('File=...' or raw '/F ...'): $DESIGNER_CONFIG_PATH" fi -declare -A SOURCE_SET_NAME_BY_TYPE=() -declare -A SOURCE_SET_PATH_BY_TYPE=() -required_types=( - CONFIGURATION - EXTENSION -) -if [[ "$BUILDER_BACKEND" == "DESIGNER" ]]; then - required_types+=( - EXTERNAL_DATA_PROCESSORS - EXTERNAL_REPORTS - ) -fi +CONFIGURATION_SOURCE_SET_NAME="" +CONFIGURATION_SOURCE_SET_PATH="" +EXTENSION_SOURCE_SET_NAME="" +EXTENSION_SOURCE_SET_PATH="" +EXTERNAL_PROCESSOR_SOURCE_SET_NAME="" +EXTERNAL_PROCESSOR_SOURCE_SET_PATH="" +EXTERNAL_REPORT_SOURCE_SET_NAME="" +EXTERNAL_REPORT_SOURCE_SET_PATH="" while IFS=$'\t' read -r source_set_name source_set_type source_set_path; do source_set_name="$(strip_shell_quotes "$source_set_name")" @@ -565,28 +666,36 @@ while IFS=$'\t' read -r source_set_name source_set_type source_set_path; do die "Each source-set must define name, type, and path: $DESIGNER_CONFIG_PATH" fi - if [[ -n "${SOURCE_SET_NAME_BY_TYPE[$source_set_type]:-}" ]]; then - die "Live Designer config must define only one source-set with type '$source_set_type': $DESIGNER_CONFIG_PATH" - fi - - SOURCE_SET_NAME_BY_TYPE["$source_set_type"]="$source_set_name" - SOURCE_SET_PATH_BY_TYPE["$source_set_type"]="$source_set_path" + case "$source_set_type" in + CONFIGURATION) + [[ -z "$CONFIGURATION_SOURCE_SET_NAME" ]] || die "Live Designer config must define only one source-set with type '$source_set_type': $DESIGNER_CONFIG_PATH" + CONFIGURATION_SOURCE_SET_NAME="$source_set_name" + CONFIGURATION_SOURCE_SET_PATH="$source_set_path" + ;; + EXTENSION) + [[ -z "$EXTENSION_SOURCE_SET_NAME" ]] || die "Live Designer config must define only one source-set with type '$source_set_type': $DESIGNER_CONFIG_PATH" + EXTENSION_SOURCE_SET_NAME="$source_set_name" + EXTENSION_SOURCE_SET_PATH="$source_set_path" + ;; + EXTERNAL_DATA_PROCESSORS) + [[ -z "$EXTERNAL_PROCESSOR_SOURCE_SET_NAME" ]] || die "Live Designer config must define only one source-set with type '$source_set_type': $DESIGNER_CONFIG_PATH" + EXTERNAL_PROCESSOR_SOURCE_SET_NAME="$source_set_name" + EXTERNAL_PROCESSOR_SOURCE_SET_PATH="$source_set_path" + ;; + EXTERNAL_REPORTS) + [[ -z "$EXTERNAL_REPORT_SOURCE_SET_NAME" ]] || die "Live Designer config must define only one source-set with type '$source_set_type': $DESIGNER_CONFIG_PATH" + EXTERNAL_REPORT_SOURCE_SET_NAME="$source_set_name" + EXTERNAL_REPORT_SOURCE_SET_PATH="$source_set_path" + ;; + esac done < <(extract_source_sets) -for source_set_type in "${required_types[@]}"; do - if [[ -z "${SOURCE_SET_NAME_BY_TYPE[$source_set_type]:-}" ]]; then - die "Live Designer config must declare a source-set with type '$source_set_type': $DESIGNER_CONFIG_PATH" - fi -done - -CONFIGURATION_SOURCE_SET_NAME="${SOURCE_SET_NAME_BY_TYPE[CONFIGURATION]}" -CONFIGURATION_SOURCE_SET_PATH="${SOURCE_SET_PATH_BY_TYPE[CONFIGURATION]}" -EXTENSION_SOURCE_SET_NAME="${SOURCE_SET_NAME_BY_TYPE[EXTENSION]}" -EXTENSION_SOURCE_SET_PATH="${SOURCE_SET_PATH_BY_TYPE[EXTENSION]}" -EXTERNAL_PROCESSOR_SOURCE_SET_NAME="${SOURCE_SET_NAME_BY_TYPE[EXTERNAL_DATA_PROCESSORS]:-}" -EXTERNAL_PROCESSOR_SOURCE_SET_PATH="${SOURCE_SET_PATH_BY_TYPE[EXTERNAL_DATA_PROCESSORS]:-}" -EXTERNAL_REPORT_SOURCE_SET_NAME="${SOURCE_SET_NAME_BY_TYPE[EXTERNAL_REPORTS]:-}" -EXTERNAL_REPORT_SOURCE_SET_PATH="${SOURCE_SET_PATH_BY_TYPE[EXTERNAL_REPORTS]:-}" +[[ -n "$CONFIGURATION_SOURCE_SET_NAME" ]] || die "Live Designer config must declare a source-set with type 'CONFIGURATION': $DESIGNER_CONFIG_PATH" +[[ -n "$EXTENSION_SOURCE_SET_NAME" ]] || die "Live Designer config must declare a source-set with type 'EXTENSION': $DESIGNER_CONFIG_PATH" +if [[ "$BUILDER_BACKEND" == "DESIGNER" ]]; then + [[ -n "$EXTERNAL_PROCESSOR_SOURCE_SET_NAME" ]] || die "Live Designer config must declare a source-set with type 'EXTERNAL_DATA_PROCESSORS': $DESIGNER_CONFIG_PATH" + [[ -n "$EXTERNAL_REPORT_SOURCE_SET_NAME" ]] || die "Live Designer config must declare a source-set with type 'EXTERNAL_REPORTS': $DESIGNER_CONFIG_PATH" +fi WORK_BASE_PATH="$OUTPUT_ROOT/workspace/project-root" WORK_CONFIG_PATH="$WORK_BASE_PATH/v8project.yaml" @@ -605,16 +714,26 @@ mkdir -p \ "$WORK_BASE_PATH" \ "$OUTPUT_ROOT/artifacts/external-processor" \ "$OUTPUT_ROOT/artifacts/external-report" \ + "$OUTPUT_ROOT/json" \ "$OUTPUT_ROOT/launch" cp -R "$FIXTURE_BASE_PATH/." "$WORK_BASE_PATH/" materialize_live_config "$DESIGNER_CONFIG_PATH" "$WORK_CONFIG_PATH" "$OUTPUT_ROOT" "$WORK_BASE_PATH" DESIGNER_CONFIG_PATH="$WORK_CONFIG_PATH" - -for source_set_type in "${required_types[@]}"; do - source_set_path="${SOURCE_SET_PATH_BY_TYPE[$source_set_type]}" - if [[ ! -d "$WORK_BASE_PATH/$source_set_path" ]]; then +WORK_PATH="$(extract_yaml_scalar "workPath")" + +rm -f \ + "$WORK_BASE_PATH/$CONFIGURATION_SOURCE_SET_PATH/ConfigDumpInfo.xml" \ + "$WORK_BASE_PATH/$EXTENSION_SOURCE_SET_PATH/ConfigDumpInfo.xml" +assert_source_has_no_cdfi + +for source_set_path in \ + "$CONFIGURATION_SOURCE_SET_PATH" \ + "$EXTENSION_SOURCE_SET_PATH" \ + "$EXTERNAL_PROCESSOR_SOURCE_SET_PATH" \ + "$EXTERNAL_REPORT_SOURCE_SET_PATH"; do + if [[ -n "$source_set_path" && ! -d "$WORK_BASE_PATH/$source_set_path" ]]; then die "Configured source-set path does not exist under fixture project root: $source_set_path" fi done @@ -639,6 +758,8 @@ print_stage "build incremental no-op" run_cli_json_to_file "$incremental_build_json" build assert_json_step_ok "$incremental_build_json" "$CONFIGURATION_SOURCE_SET_NAME" assert_json_step_ok "$incremental_build_json" "$EXTENSION_SOURCE_SET_NAME" +assert_source_has_no_cdfi +assert_private_cdfi_exists if [[ "$BUILDER_BACKEND" == "DESIGNER" ]]; then partial_candidate="$WORK_BASE_PATH/$CONFIGURATION_SOURCE_SET_PATH/CommonModules/ОбщийМодуль1/Ext/Module.bsl" @@ -705,6 +826,8 @@ if [[ "$BUILDER_BACKEND" == "DESIGNER" ]]; then echo "READY: $artifact" done + run_designer_runtime_state_steps + print_stage "launch smoke" run_launch_smoke else @@ -716,9 +839,5 @@ else run_cli dump --mode partial --object Catalog.Items fi -if [[ "$DESIGNER_SMOKE_PROFILE" == "extended" ]]; then - run_extended_steps -fi - echo echo "Live CLI $BUILDER_BACKEND smoke completed successfully." diff --git a/spec/acceptance/real-environment-validation.md b/spec/acceptance/real-environment-validation.md index 956cebf..e1de854 100644 --- a/spec/acceptance/real-environment-validation.md +++ b/spec/acceptance/real-environment-validation.md @@ -11,6 +11,7 @@ 3. `test` Rust/CLI/MCP-контракта 4. `package` 5. `deploy-ready artifacts` +6. `full/incremental/partial dump` через private shadow с проверкой no-clobber Под `deploy-ready artifacts` в этом репозитории понимается только публикация и проверка наличия/непустоты следующих файлов: @@ -45,7 +46,7 @@ bash scripts/test/ci-rust.sh - `V8_RUNNER_CI_SCOPE=contract` или `full` запускает `cargo test --locked` - `V8_RUNNER_CI_SCOPE=runtime-locks` запускает только lock-focused regression subset -- `V8_RUNNER_CI_SCOPE=happy-path` запускает обязательную цепочку `build -> syntax/check -> test -> package -> deploy-ready artifacts` +- `V8_RUNNER_CI_SCOPE=happy-path` запускает обязательную цепочку `build -> syntax/check -> test -> package -> deploy-ready artifacts -> private dump/no-clobber` ### 2. Mandatory Linux/Windows happy-path @@ -75,6 +76,10 @@ V8_RUNNER_CI_SCOPE=happy-path bash scripts/test/ci-rust.sh 7. `test` 8. `make` для `.cf/.cfe/.epf/.erf` 9. проверку, что все deploy-ready артефакты существуют и не пусты +10. отсутствие `ConfigDumpInfo.xml` в source и наличие private CDFI под `workPath/ib-state/v1` +11. успешный `dump full` bootstrap в пустой owned target, затем `dump incremental` и + `dump partial` с детерминированным `LocalOnly.txt` conflict, точным receipt и byte-exact + сохранением конфликтующего source/runtime state Partial smoke contract: @@ -125,7 +130,8 @@ bash scripts/test/live-cli-fixture.sh - `V8TR_BIN` - путь к бинарю `v8-runner` - `V8TR_PLATFORM_PATH` - явный override пути до `1cv8`/`1cv8.exe` -- `V8TR_DESIGNER_SMOKE_PROFILE=mandatory|extended` - mandatory по умолчанию; `extended` включает dump-only хвост +- `V8TR_DESIGNER_SMOKE_PROFILE=mandatory|extended` - `mandatory` по умолчанию; `extended` + сохранён как совместимый alias, а private dump/no-clobber contour обязателен для обоих профилей - `V8TR_DESIGNER_TEST_MODE=none|va|yaxunit-all|module` - явный запуск 1С test-stage helper-а; `none` по умолчанию - `V8TR_DESIGNER_TEST_MODULE` - обязателен при `V8TR_DESIGNER_TEST_MODE=module` - `V8TR_DESIGNER_ALLOW_MISSING_CONFIG=1` - разрешить `SKIPPED` вместо hard failure только для non-blocking/soft-skip контекстов @@ -149,6 +155,11 @@ Cross-platform hardening: Критерий успеха: - все стадии `build -> syntax/check -> test -> package -> deploy-ready artifacts` завершаются с `exit code 0` +- full dump успешно публикует declarative source из private shadow в пустой owned target; + incremental/partial dump возвращают ожидаемый conflict exit code `3`, exact receipt указывает + только `LocalOnly.txt` и не содержит processed targets +- source и `ib-state/v1` после incremental/partial conflict остаются byte-exact, CDFI присутствует + только в private runtime state, а общий smoke завершается с `exit code 0` - существуют и не пусты: - `target/manual-tests/live-cli-designer/artifacts/configuration.cf` - `target/manual-tests/live-cli-designer/artifacts/extension.cfe` @@ -201,13 +212,13 @@ Windows runner contract for this helper layer is explicit: ## Матрица покрытия -| Контур | Linux | Windows | Blocking | Build | Syntax/check | Test | Package | Deploy-ready artifacts | -| --- | --- | --- | --- | --- | --- | --- | --- | --- | -| `ci-rust contract` | yes | yes | yes | Rust | Rust | Rust | no | no | -| `ci-rust happy-path` | yes | yes | yes on trusted | Rust + real 1C | real | Rust by default; real 1C opt-in | real | real | -| `live-mcp-http` | optional | optional | no | real via MCP | real via MCP | real via MCP | n/a | n/a | -| `live-cli-ibcmd` | optional | optional | no | real (`IBCMD`) | n/a | n/a | diagnostic dump/export only | n/a | -| `live-cli-designer` | optional | optional | no | real (`DESIGNER`) | real | real opt-in | real | real | +| Контур | Linux | Windows | Blocking | Build | Syntax/check | Test | Package | Dump/no-clobber | Deploy-ready artifacts | +| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | +| `ci-rust contract` | yes | yes | yes | Rust | Rust | Rust | no | no | no | +| `ci-rust happy-path` | yes | yes | yes on trusted | Rust + real 1C | real | Rust by default; real 1C opt-in | real | real | real | +| `live-mcp-http` | optional | optional | no | real via MCP | real via MCP | real via MCP | n/a | n/a | n/a | +| `live-cli-ibcmd` | optional | optional | no | real (`IBCMD`) | n/a | n/a | n/a | diagnostic dump/export | n/a | +| `live-cli-designer` | optional | optional | no | real (`DESIGNER`) | real | real opt-in | real | real | real | ## Ограничения и TODO hooks diff --git a/spec/architecture/arc42/02-constraints.md b/spec/architecture/arc42/02-constraints.md index c48a17a..9e425b4 100644 --- a/spec/architecture/arc42/02-constraints.md +++ b/spec/architecture/arc42/02-constraints.md @@ -10,11 +10,11 @@ - Для реализованных сценариев `builder=IBCMD` уже поддерживает file и server ИБ; server path требует полный `infobase.dbms.kind/server/name`, а оставшиеся ограничения считаются gaps, а не целевой нормой. - Все инструменты должны проектироваться с целевой поддержкой серверных информационных баз, если соответствующая операция платформы 1С принципиально поддерживает серверное подключение. - MCP реализован на `rmcp`, `tokio` и `axum`. -- Состояние отслеживания изменений хранится в `workPath/hash-storages/*.redb`. +- Состояние отслеживания изменений хранится в versioned per-IB/per-source layout `workPath/ib-state/v1`; legacy `hash-storages` не мигрируется. - `workPath` является owned runtime root; публичные CLI/MCP команды, которые читают или пишут runtime state под ним, должны владеть workspace lock. - MCP execution admission и HTTP session capacity являются разными лимитами и не заменяют workspace lock. - Публичная поверхность MCP намеренно уже, чем CLI: например, `init` и `extensions` не публикуются как MCP tools. -- Full replacement `dump` и `artifacts` должны публиковаться через staging/backup рядом с target; incremental/partial dump остаются non-atomic update modes. +- Все dump-режимы используют private shadow и recoverable managed-file journal; full-replacement artifacts публикуются через staging/backup рядом с target. - Единая timeout/cancellation policy для CLI и MCP является целевым контрактом. Текущие implementation gaps фиксируются в ADR-0014 и не должны становиться новой нормой. ### 2.2 Организационные и продуктовые ограничения diff --git a/spec/architecture/arc42/04-solution-strategy.md b/spec/architecture/arc42/04-solution-strategy.md index 1028131..936d9c5 100644 --- a/spec/architecture/arc42/04-solution-strategy.md +++ b/spec/architecture/arc42/04-solution-strategy.md @@ -7,12 +7,14 @@ - CLI и MCP остаются тонкими адаптерами над транспортно-нейтральными use case. - `v8project.yaml` и typed `AppConfig` являются главным конфигурационным контрактом; unsafe/unsupported combinations должны отклоняться на validation boundary. - Прямое взаимодействие с инструментами 1С инкапсулировано в выделенных платформенных адаптерах. -- Анализ изменений используется для предпочтения инкрементальной работы вместо полного rebuild. +- Анализ изменений использует versioned per-IB/per-source state под `workPath/ib-state/v1`; legacy state не мигрируется, а отсутствие scoped state означает full bootstrap. - Структурированные типы результатов сохраняются до границы адаптера, а затем рендерятся отдельно для CLI и MCP. - MCP рассматривается не только как транспорт: он добавляет сессии, параллелизм, нормализацию, admission control и обработку транспортных ошибок. - Публичные команды над одним canonical `workPath` сериализуются через workspace lock; nested flows используют явные unlocked entrypoints только под внешним lock. - Timeout/cancellation реализуются поверх общего execution core по host-specific policy: MCP может отсоединять caller от running EDT работы и удерживать capacity до terminal state, а CLI blocking flows ждут terminal cleanup или принудительно закрывают свой shared EDT manager перед возвратом. -- Full replacement `dump` и `artifacts` публикуются через staging/backup, чтобы platform failure до publish сохранял старый target. +- Designer build загружает private source staging с private CDFI и не передаёт platform живое source tree. +- Любой dump выполняется в private shadow и публикует managed files через B/S/D merge и recoverable journal; package/external artifacts сохраняют sibling staging/backup contract. +- Source publication и runtime-state generation связываются точным UUID transaction token, поэтому recovery не принимает чужую одноимённую generation за успех. - `tools download` остаётся CLI-only bootstrap-сценарием: он материализует внешние release assets в рабочие каталоги проекта и обновляет local overlay, но не расширяет MCP surface и не выполняет platform load в ИБ. - Runner-like сценарии используют общий execution grammar: pipeline vocabulary, step entries и `ExecutionOutcome` как canonical domain outcome. - Общий интерактивный EDT actor вынесен в `platform::edt_session` и переиспользуется и MCP `check_syntax_edt`, и CLI interactive EDT use cases; различается только host policy (MCP может prewarm shared host, CLI остаётся lazy и short-lived). diff --git a/spec/architecture/arc42/05-building-block-view.md b/spec/architecture/arc42/05-building-block-view.md index bd08a34..f777ad8 100644 --- a/spec/architecture/arc42/05-building-block-view.md +++ b/spec/architecture/arc42/05-building-block-view.md @@ -6,13 +6,13 @@ - `cli`: разбор аргументов и CLI-специфичное представление результатов. - `config`: загрузка typed YAML-контракта, defaults и ранняя валидация unsafe combinations. -- `use_cases`: транспортно-нейтральная оркестрация, контекст выполнения, workspace lock helpers и command-specific flows. +- `use_cases`: транспортно-нейтральная оркестрация, private state/shadow transactions, workspace lock helpers и command-specific flows. - `mcp`: MCP DTO, сервисная граница, транспорты, параллелизм и управление сессиями. - `platform`: поиск внешних инструментов по версии/маске и выполнение команд против утилит 1С. - `change_detection`: инкрементальный анализ и сохранённое файловое состояние. - `parsers`: преобразование сырых логов и отчётов в структурированные результаты. -- `domain` и `output`: общие модели результатов, `ExecutionOutcome` и CLI-примитивы представления. -- `support`: сквозные утилиты для файловой системы, staging/backup publication, логирования, temp и ошибок. +- `domain` и `output`: runtime identities, exact sync receipts, typed transaction token, общие модели результатов и CLI-примитивы представления. +- `support`: сквозные filesystem utilities, включая Windows-specific claim/replace/remove, staging/backup publication, логирование, temp и ошибки. ```mermaid flowchart TB @@ -47,6 +47,7 @@ flowchart TB - Для runner-like сценариев собирает typed pipeline-like flow и заполняет `ExecutionOutcome` вместо нового ad hoc result shape. - Для `convert` выводит direction из `format`, резолвит `source-set` из `v8project.yaml` и публикует generated output либо под default `workPath/convert/out`, либо под explicit `--output` root с mirror-layout. - Для `tools download ` получает latest release metadata выбранного инструмента, скачивает sources/artifact, обновляет `v8project.local.yaml` для Vanessa/client MCP и при `yaxunit --sources` добавляет YAxUnit как project `source-set` `tests`. +- `runtime_state` готовит и восстанавливает CDFI/baseline/observation как одну generation; `dump_shadow`, `shadow_merge`, `source_publication` и `dump_execution` реализуют private dump и recoverable manifest publication. #### `mcp` @@ -69,6 +70,7 @@ flowchart TB - Отслеживает хеши и timestamp. - Группирует изменения по логическим `source-set`. - Даёт оркестратору не просто список файлов, а сигнал для выбора partial/full стратегии. +- Хранит `redb` snapshot по разрешённому per-IB/per-source path под `workPath/ib-state/v1`, а не в legacy global layout. #### `parsers` @@ -77,9 +79,11 @@ flowchart TB #### `domain` и `output` - `domain` фиксирует общие структуры результата, включая `ExecutionOutcome`, `ExecutionStatus`, `ExecutionError`, metrics, artifacts и минимальный `StepResult`. +- `domain::runtime_state` фиксирует secret-free infobase/source identities и canonical UUID dump transaction token; `domain::sync_receipt` валидирует exact audit records. - `output` содержит только presentation-layer примитивы и не должен становиться бизнес-слоем. #### `support` - Содержит filesystem helpers для atomic-like replacement через staging/backup. - Хранит metadata sidecars для staging/backup cleanup и не должен превращать internal temp naming в публичный API. +- `support::windows_fs` реализует handle-safe claim/replace/remove primitives; их обязательный gate — contract tests на `windows-latest`, результат которых нельзя подменять Unix-only локальным запуском. diff --git a/spec/architecture/arc42/06-runtime-view.md b/spec/architecture/arc42/06-runtime-view.md index 2884466..fccd00a 100644 --- a/spec/architecture/arc42/06-runtime-view.md +++ b/spec/architecture/arc42/06-runtime-view.md @@ -8,20 +8,25 @@ sequenceDiagram participant Adapter as Адаптер CLI/MCP participant UC as Use case сборки participant CD as Анализ изменений + participant Stage as Private source transaction participant PF as Платформенный адаптер participant IB as Информационная база 1С + participant State as Per-IB runtime state User->>Adapter: запрос build Adapter->>UC: нормализованный запрос UC->>CD: определить изменённые source-set - CD-->>UC: изменённые файлы и подсказки по режиму + CD-->>UC: bootstrap/изменения и подсказки по режиму alt Изменений нет UC-->>Adapter: skipped/success результат else Найдены изменения - UC->>PF: выполнить загрузку через Designer или IBCMD + UC->>Stage: копировать managed source, исключить source CDFI/symlinks/workPath + UC->>Stage: добавить только валидный private CDFI + UC->>PF: выполнить загрузку из private staging PF->>IB: import/apply изменений IB-->>PF: результат платформы PF-->>UC: структурированный итог исполнения + UC->>State: commit CDFI/baseline/observation generation UC-->>Adapter: результат build end ``` @@ -31,7 +36,8 @@ sequenceDiagram - Public CLI/MCP boundary должен владеть workspace lock для canonical `workPath` до dispatch use case. - `CONFIGURATION` обрабатывается раньше расширений. - Выбор между partial и full строится по анализу изменений и возможностям backend. -- Состояние сохраняется только после успешного выполнения. +- Legacy state не мигрируется: отсутствие scoped state или missing/corrupt private CDFI означает full bootstrap, даже для пустого source-set. +- Platform не получает живое source tree; состояние сохраняется только после успешного pipeline. - Для EDT source-set export decision и generated Designer load decision используют разные change-detection contexts. ### 6.2 Сценарий `test` @@ -102,18 +108,49 @@ sequenceDiagram - Ожидание в очереди, baseline reset/probe и выполнение команды используют один и тот же ограниченный бюджет таймаута. - Host policy различается: MCP может отпустить caller после running cancel/timeout и дождаться terminal state асинхронно внутри shared actor, а CLI blocking adapter ждёт terminal cleanup или завершает собственный short-lived manager принудительно перед возвратом. -### 6.6 Full Replacement `dump` / `artifacts` Publication +### 6.6 `dump` И `artifacts` Publication + +`dump` использует отдельный от artifacts контракт: + +```mermaid +sequenceDiagram + participant UC as Dump use case + participant Shadow as Private shadow + participant Source as Project source + participant Journal as Source journal + participant State as Per-IB state + + UC->>Journal: recover exact generation + transaction token + UC->>Shadow: full/incremental/partial platform dump + UC->>UC: построить B/S/D plan и revalidate source + alt conflict или TOCTOU mismatch + UC-->>Source: no writes + UC-->>State: no generation advance + else conflict-free + UC->>Journal: backup + ordered managed-file actions + Journal->>Source: publish manifest + UC->>State: commit generation с тем же UUID token + UC->>Journal: mark state visible exact (generation, token) + end +``` + +- `B=absent, S=present, D=absent` является conflict: runner не удаляет не принадлежавший baseline локальный файл. +- Любой conflict блокирует весь source-set; `ConfigDumpInfo.xml` и unmanaged entries не публикуются. +- Receipt lists являются audit dimensions, не partition: в `applied` одинаковый target может одновременно быть `processed` и `skipped`. +- Recovery завершает transaction вперёд только при exact generation/token match; чужая transaction с тем же номером generation ведёт к rollback source changes. + +Package/external artifacts используют full-replacement staging/backup: ```mermaid sequenceDiagram participant User as CLI пользователь - participant UC as Use case dump/artifacts + participant UC as Use case artifacts participant PF as Platform adapter participant Stage as Sibling staging path participant Target as User target path participant Backup as Sibling backup path - User->>UC: dump/artifacts request + User->>UC: artifacts request UC->>Stage: подготовить staging рядом с target UC->>PF: записать результат в staging PF-->>UC: platform result @@ -137,7 +174,6 @@ sequenceDiagram - Staging и backup находятся рядом с target, чтобы не переходить границу файловой системы при rename. - Orphan cleanup может удалять только stale staging/backup paths с metadata `tool=v8-runner` и matching target identity. -- `dump incremental` и `dump partial` не получают full replacement guarantee и остаются non-atomic update modes. - Publication phase после переноса старого target в backup является filesystem critical phase. ### 6.7 Command Boundary, Admission и Cancellation diff --git a/spec/architecture/arc42/08-cross-cutting-concepts.md b/spec/architecture/arc42/08-cross-cutting-concepts.md index 2ae8508..3fc91ce 100644 --- a/spec/architecture/arc42/08-cross-cutting-concepts.md +++ b/spec/architecture/arc42/08-cross-cutting-concepts.md @@ -48,13 +48,13 @@ ### 8.4.1 Публикация dump/artifacts -- Full replacement dump/artifacts сначала пишутся в staging path рядом с target. +- Full-replacement artifacts сначала пишутся в staging path рядом с target. - При замене существующего target старое состояние временно переносится в backup и используется для rollback при publish failure. - Cleanup backup/staging после успешной публикации выполняется best-effort и может вернуться как warning. - Staging/backup cleanup опирается на metadata sidecar: `tool`, `kind`, `run_id`, `target_path`, `target_identity`, `created_at`. - Orphan cleanup не должен удалять malformed, foreign или recent temp paths. -- Incremental/partial dump остаются non-atomic update modes. -- Правила staging/backup publication описаны в ADR-0015. +- Все dump-режимы выполняются в private shadow и публикуют managed-file manifest через recoverable journal; B/S/D conflict блокирует весь source-set. +- Правила artifacts staging/backup описаны в ADR-0015, dump publication и exact generation/token recovery — в ADR-0023. ### 8.5 Параллелизм и таймауты diff --git a/spec/architecture/arc42/09-architecture-decisions.md b/spec/architecture/arc42/09-architecture-decisions.md index 2c88504..2e85a99 100644 --- a/spec/architecture/arc42/09-architecture-decisions.md +++ b/spec/architecture/arc42/09-architecture-decisions.md @@ -18,12 +18,15 @@ | [ADR-0012: On-demand change detection и файловая partial-load стратегия](../../decisions/0012-on-demand-change-detection-i-faylovaya-partial-load-strategiya.md) | `accepted`, `2026-04-20` | Change detection запускается on-demand, хранит per-context `redb` snapshots и деградирует в full execution при unsafe partial cases. | | [ADR-0013: MCP execution admission, timeout/cancellation routing и HTTP session capacity](../../decisions/0013-mcp-execution-admission-timeout-cancellation-routing-i-http-session-capacity.md) | `accepted`, `2026-04-20` | MCP execution admission, timeout/cancellation routing и HTTP session capacity являются разными guardrails; MCP cancellation/deadline должны идти в общую execution policy. | | [ADR-0014: Единая timeout/cancellation policy для CLI и MCP команд](../../decisions/0014-edinaya-timeout-cancellation-policy-dlya-cli-i-mcp-komand.md) | `accepted`, `2026-04-20` | Целевой контракт: каждая public command имеет deadline, cancellation ждёт terminal state, mutating critical phases не hard-kill by default. | -| [ADR-0015: Атомарная публикация dump/artifacts через staging/backup](../../decisions/0015-atomarnaya-publikatsiya-dump-artifacts-cherez-staging-backup.md) | `accepted`, `2026-04-21` | Full replacement dump/artifacts публикуются через sibling staging/backup, rollback context и metadata-based orphan cleanup; incremental/partial остаются non-atomic. | +| [ADR-0015: Атомарная публикация dump/artifacts через staging/backup](../../decisions/0015-atomarnaya-publikatsiya-dump-artifacts-cherez-staging-backup.md) | `accepted`, `2026-04-21` | Full-replacement artifacts публикуются через sibling staging/backup; dump publication уточнена ADR-0023. | | [ADR-0016: Единый `ExecutionOutcome` и pipeline steps для runner-like сценариев](../../decisions/0016-edinyy-executionoutcome-i-pipeline-steps-dlya-runner-like-stsenariev.md) | `accepted`, `2026-04-21` | Runner-like и pipeline-like сценарии используют `ExecutionOutcome` как canonical domain outcome и общую vocabulary pipeline blocks/steps. | | [ADR-0017: `v8project.yaml` / `source-set` как главный конфигурационный контракт](../../decisions/0017-v8project-yaml-source-set-kak-glavnyy-konfiguratsionnyy-kontrakt.md) | `accepted`, `2026-04-20` | `v8project.yaml` -> `AppConfig` -> `config::validate` является главным config contract; `source-set[].type`, `source-set.name` и `workPath` задают runtime identity. | | [ADR-0018: Перенести контракт информационной базы в `infobase`](../../decisions/0018-perenesti-kontrakt-informatsionnoy-bazy-v-infobase.md) | `accepted`, `2026-04-21` | `infobase.connection` и `infobase.user/password` заменяют top-level `connection`/`credentials`; `infobase.dbms` задаёт DBMS-level contract для `IBCMD` server connection. | | [ADR-0019: Обеспечивать наличие серверной ИБ через `ibcmd` в `init`](../../decisions/0019-sozdavat-servernuyu-infobazu-cherez-ibcmd-pri-init-pri-otsutstvii.md) | `accepted`, `2026-04-22` | Для `builder=IBCMD` + server connection `init` использует `ibcmd infobase create --create-database` как ensure-step; отдельный pre-check наличия не обязателен, а benign `already exists` нормализуется как non-error outcome. | | [ADR-0020: Упростить CLI-only `convert` до repo-aware конвертации текущих исходников проекта](../../decisions/0020-dobavit-cli-only-convert-dlya-dvustoronney-konvertatsii-edt-i-designer.md) | `accepted`, `2026-04-22` | Фиксирует и уже реализует repo-aware `convert [--source-set ] [--output ]`, который работает от `v8project.yaml`, выводит направление из `format`, публикует output под default `workPath/convert/out` или explicit mirror-layout root и не выносит low-level EDT flags в public surface. | +| [ADR-0021: Ввести локальный overlay для `v8project.yaml`](../../decisions/0021-lokalnyy-overlay-config.md) | `accepted`, `2026-05-02` | Optional `v8project.local.yaml` автоматически накладывает локальные paths, credentials и tool-настройки на primary project config до CLI overrides. | +| [ADR-0022: Ввести общий механизм подготовки расширений и использовать его для `client_mcp`](../../decisions/0022-universalnyy-mehanizm-podgotovki-rasshireniy-i-client-mcp-extension.md) | `accepted`, `2026-05-02` | Tool extensions готовятся общим source/artifact pipeline вне project `source-set`; source-вариант использует on-demand change detection. | +| [ADR-0023: Изолировать runtime state по информационной базе и использовать private shadow](../../decisions/0023-izolirovat-runtime-state-po-infobase-i-ispolzovat-private-shadow.md) | `accepted`, `2026-07-21` | Per-IB `ib-state/v1`, private build/dump shadows, B/S/D merge и exact generation/token recovery защищают source tree и incremental decisions. | Архитектурные инварианты для агентов и контрибьюторов зафиксированы в [spec/architecture/invariants.md](../invariants.md). @@ -32,12 +35,12 @@ - Public surface changes нужно оценивать отдельно для CLI и MCP: наличие CLI-команды не означает доступность MCP tool. - `convert` является осознанной CLI-only командой и не должен трактоваться как автоматический кандидат в MCP tool. - Use case layer остаётся общей транспортно-нейтральной orchestration boundary, а adapters отвечают за presentation, DTO и transport/runtime failures. -- `source-set.name` и canonical `workPath` являются runtime identity. Изменения naming/path rules затрагивают config validation, change detection, generated directories и workspace lock. +- Secret-free infobase identity, source identity и canonical `workPath` определяют runtime state; `source-set.name` остаётся diagnostic/path input, но не является достаточной identity само по себе. - `infobase` является единственным config contract для строки подключения, пользователя ИБ и DBMS-level доступа; top-level `connection`/`credentials` не поддерживаются. - Полный `infobase.dbms` contract при `builder=IBCMD` достаточно явно разрешает server infobase provisioning в `init`; отдельный top-level provisioning flag для этого не требуется. -- Repo-aware `convert` и reverse sync из ИБ в файлы — разные сценарии; `dump format=EDT` реализован как отдельный flow поверх internal Designer snapshot и EDT import, а не как alias или скрытый sub-step `convert`. +- Repo-aware `convert` и reverse sync из ИБ в файлы — разные сценарии; `dump format=EDT` использует private Designer/configured-source shadows и EDT import, а не `convert` или live generated target. - MCP concurrency имеет два независимых контура: execution admission для tool calls и HTTP session capacity для stateful transport lifecycle. -- Target publication safety не обеспечивается workspace lock: full replacement outputs требуют staging/backup contract рядом с target. +- Target publication safety не обеспечивается workspace lock: artifacts требуют sibling staging/backup, а dump — private shadow и recoverable manifest journal. - ADR-0014 и ADR-0016 описывают целевую архитектуру с известными migration gaps. Новые команды должны следовать этим контрактам, даже если часть старых сценариев ещё находится в переходном состоянии. ### Правила актуализации diff --git a/spec/architecture/arc42/11-risks-and-technical-debt.md b/spec/architecture/arc42/11-risks-and-technical-debt.md index e01b3e8..01084af 100644 --- a/spec/architecture/arc42/11-risks-and-technical-debt.md +++ b/spec/architecture/arc42/11-risks-and-technical-debt.md @@ -3,7 +3,7 @@ - Журнал ADR уже введён, но его нужно поддерживать синхронно с кодом и публичной документацией. - Публичная и внутренняя документация могут расходиться, если их не обновлять вместе с кодом. - Общий shared interactive EDT-path теперь вынесен в `platform`, но остаётся риск регрессии к третьему публичному execution path, если новые EDT-сценарии начнут обходить общий actor/manager или документация/tests перестанут держать инварианты ADR-0007. -- `dump format=EDT` теперь зависит от внутреннего Designer snapshot под `workPath/designer/`; новые изменения не должны подменять этот reverse-sync path командой `convert` или обходить staged publication target-каталога. +- `dump format=EDT` использует private per-IB Designer/configured-source shadows и B/S/D publication; новые изменения не должны подменять этот reverse-sync path командой `convert` или обходить manifest journal/conflict gate. - Поддержка `IBCMD` остаётся уже, чем поддержка Designer. - Provisioning contract из ADR-0019 реализован только для `builder=IBCMD`; `builder=DESIGNER` по-прежнему пропускает server infobase create step и это остаётся документированным ограничением. - Общая timeout/cancellation policy из ADR-0014 является целевой архитектурой и ещё не полностью реализована во всех public commands. diff --git a/spec/architecture/invariants.md b/spec/architecture/invariants.md index a93e422..5053f0e 100644 --- a/spec/architecture/invariants.md +++ b/spec/architecture/invariants.md @@ -82,15 +82,17 @@ ## Dump And Artifacts Publication -1. Full-replacement `dump` и `artifacts` publication не должны писать напрямую в существующий target. -2. Full dump и package/external artifacts publication должны идти через staging path рядом с target и backup старого target. -3. Platform failure до publish должен сохранять старый target. -4. Publish failure должен пытаться rollback backup -> target и surfaced rollback context, если восстановление не удалось. -5. Cleanup backup/staging после успешного publish выполняется best-effort; cleanup failure становится warning/degraded success, а не failed publish. -6. `dump incremental` и `dump partial` являются non-atomic update modes и не получают staging replacement guarantee. -7. Orphan cleanup должен удалять только stale v8-runner staging/backup paths с matching target identity. - -См. [ADR-0015](../decisions/0015-atomarnaya-publikatsiya-dump-artifacts-cherez-staging-backup.md). +1. Package/external artifacts full replacement не должен писать напрямую в существующий target и проходит через sibling staging/backup. +2. Любой `dump` (`full`, `incremental`, `partial`) выполняет platform export только в private shadow; platform не получает project source как dump target. +3. Dump publication работает по managed-file manifest и не заменяет source root целиком. +4. B/S/D conflict или TOCTOU mismatch блокирует весь source-set до первой записи и не продвигает private generation. +5. Source journal хранит byte-exact backups и обеспечивает restart rollback; мгновенная multi-file filesystem atomicity не обещается. +6. Source publication и runtime-state commit связывает точная пара `(generation, DumpTransactionId)`; совпавшей generation с другим token недостаточно для forward recovery. +7. `ConfigDumpInfo.xml`, symlinks, nested `workPath` и unmanaged entries не публикуются в source tree. +8. Artifacts publish failure должен пытаться rollback backup -> target; cleanup после success выполняется best-effort. +9. Orphan cleanup должен удалять только stale v8-runner staging/backup paths с matching target identity. + +См. [ADR-0015](../decisions/0015-atomarnaya-publikatsiya-dump-artifacts-cherez-staging-backup.md) и [ADR-0023](../decisions/0023-izolirovat-runtime-state-po-infobase-i-ispolzovat-private-shadow.md). ## Pipeline Execution Outcome @@ -118,14 +120,16 @@ ## Change Detection And Partial Load 1. Change detection выполняется on-demand во время build/export/load decision, без background watcher. -2. Persistent state хранится в per-context `redb` storages под `workPath/hash-storages`. -3. Для `format=DESIGNER` используется один `designer-` context на source-set. -4. Для `format=EDT` используются два context на source-set: `edt-` для export decision и `designer-` для load decision. -5. Recoverable scan/storage ошибки должны деградировать в full execution или full rescan; hard storage и concurrent generation errors должны surfaced as failures. -6. Partial load является conservative file-level strategy: `Configuration.xml`, deletions, unsafe expansion, empty expanded set или превышение threshold ведут к full load. -7. Prepared snapshot коммитится только после successful platform export/load step. - -См. [ADR-0012](../decisions/0012-on-demand-change-detection-i-faylovaya-partial-load-strategiya.md). +2. Persistent state хранится только под `workPath/ib-state/v1//-`; fingerprints opaque и не редактируются вручную. +3. Legacy `workPath/hash-storages` не мигрируется и не переиспользуется; отсутствие scoped state означает full bootstrap, а не `NoChanges`. +4. Логические Designer, EDT и tool-extension contexts изолируются по ИБ, source identity, format, backend и role. +5. Designer build работает с private source transaction: source CDFI игнорируется, валидный private CDFI может быть seeded, а platform никогда не меняет source tree. +6. CDFI, baseline и source observation становятся видимыми одной recoverable generation только после successful platform pipeline. +7. Recoverable scan/storage ошибки должны деградировать в full execution или full rescan; hard storage и concurrent generation errors должны surfaced as failures. +8. Partial load является conservative file-level strategy: `Configuration.xml`, deletions, unsafe expansion, empty expanded set или превышение threshold ведут к full load. +9. Receipt lists являются независимыми audit dimensions: одинаковая запись может быть одновременно `processed` и `skipped` для applied operation; inconsistent overlap запрещён. + +См. [ADR-0012](../decisions/0012-on-demand-change-detection-i-faylovaya-partial-load-strategiya.md) и [ADR-0023](../decisions/0023-izolirovat-runtime-state-po-infobase-i-ispolzovat-private-shadow.md). ## Shared EDT diff --git a/spec/decisions/0002-izolirovat-runtime-state-po-source-set-pod-workpath.md b/spec/decisions/0002-izolirovat-runtime-state-po-source-set-pod-workpath.md index 4879ae8..5773ab7 100644 --- a/spec/decisions/0002-izolirovat-runtime-state-po-source-set-pod-workpath.md +++ b/spec/decisions/0002-izolirovat-runtime-state-po-source-set-pod-workpath.md @@ -3,6 +3,12 @@ - Статус: `accepted` - Дата: `2026-04-20` +> Уточнено [ADR-0023](0023-izolirovat-runtime-state-po-infobase-i-ispolzovat-private-shadow.md): +> runtime state теперь дополнительно изолируется по ИБ и source identity; старый +> `workPath/hash-storages` и имена каталогов `designer-*` / `edt-*` не являются +> целевым persisted layout. Описанные ниже логические Designer/EDT contexts сохраняют +> смысл, но физически разрешаются в versioned per-IB state. + ## Контекст `v8-runner` выполняет операции над основной конфигурацией, расширениями, EDT-проектами и сгенерированными Designer-файлами. @@ -87,32 +93,32 @@ EDT source-set 1. `src/config/model.rs` описывает `source-set`, `workPath`, `format` и `builder`. 2. `src/config/validate.rs` валидирует уникальность и безопасность `source-set` и рабочие ограничения. -3. `src/change_detection/source_sets.rs` создает контексты `designer-` и `edt-`. +3. `src/change_detection/source_sets.rs` создает логические Designer/EDT contexts и разрешает их в per-IB runtime state из ADR-0023. 4. `src/change_detection/hash_storage.rs` хранит состояние в `redb`. 5. `src/change_detection/partial_load.rs` принимает partial/full decision по Designer-файлам. 6. `src/use_cases/build_project.rs` использует `edt-*` context для export decision и `designer-*` context для load decision. 7. EDT build состоит из двух независимых последовательных стадий: - EDT stage анализирует `edt-` и при необходимости выполняет export; - - после успешного export коммитится только `edt-*` snapshot; + - после успешного export EDT observation остаётся prepared до завершения Designer stage; - Designer stage всегда анализирует `designer-` после успешной или skipped EDT stage, потому что прошлый load/apply мог быть отменён или сломан; - Designer/IBCMD load/apply выполняется только если Designer analysis нашёл изменения; - - `designer-*` snapshot коммитится только после successful load/apply; + - EDT observation, `designer-*` snapshot, baseline и CDFI публикуются согласованной generation только после successful load/apply; - ошибка на предыдущей стадии запрещает переход к следующей стадии. При дальнейших изменениях: -1. новые сценарии, которым нужно состояние изменений, должны использовать per-source-set context, а не глобальный storage; +1. новые сценарии, которым нужно состояние изменений, должны использовать per-IB/per-source identity, а не глобальный storage; 2. новые runtime-артефакты должны размещаться под `workPath`; 3. EDT-сценарии должны явно выбирать, работают они с `edt-*` context или `designer-*` context. ## Верификация -- [x] Для `format=DESIGNER` существует один Designer context на `source-set`. -- [x] Для `format=EDT` существуют два context на `source-set`: `edt-` и `designer-`. +- [x] Для `format=DESIGNER` существует один логический Designer context на `source-set`. +- [x] Для `format=EDT` существуют два логических context на `source-set`: EDT source и generated Designer source. - [x] Generated Designer output для EDT находится под `workPath/designer/`. - [x] Partial/full load decision выполняется по Designer-format context. -- [x] `redb` storage используется как per-context persisted state, а не как единый глобальный индекс. -- [ ] EDT build коммитит `edt-*` snapshot сразу после successful export. -- [ ] EDT build запускает Designer analysis после successful или skipped EDT stage независимо от наличия EDT-изменений. -- [ ] EDT build коммитит `designer-*` snapshot только после successful Designer/IBCMD load/apply. -- [ ] Ошибка на EDT stage останавливает pipeline до Designer stage. +- [x] `redb` storage используется как per-IB/per-source persisted state, а не как единый глобальный индекс. +- [x] EDT observation и Designer state публикуются в согласованной recoverable generation после успешного pipeline; прежнее требование немедленного commit после export заменено ADR-0023. +- [x] EDT build запускает Designer analysis после successful или skipped EDT stage независимо от наличия EDT-изменений. +- [x] EDT build коммитит согласованную state generation только после successful Designer/IBCMD load/apply. +- [x] Ошибка на EDT stage останавливает pipeline до Designer stage. diff --git a/spec/decisions/0012-on-demand-change-detection-i-faylovaya-partial-load-strategiya.md b/spec/decisions/0012-on-demand-change-detection-i-faylovaya-partial-load-strategiya.md index 9fd21f3..d61d490 100644 --- a/spec/decisions/0012-on-demand-change-detection-i-faylovaya-partial-load-strategiya.md +++ b/spec/decisions/0012-on-demand-change-detection-i-faylovaya-partial-load-strategiya.md @@ -3,6 +3,10 @@ - Статус: `accepted` - Дата: `2026-04-20` +> Уточнено [ADR-0023](0023-izolirovat-runtime-state-po-infobase-i-ispolzovat-private-shadow.md): +> snapshots хранятся в versioned per-IB layout, а отсутствие scoped state означает +> full bootstrap, а не пустой успешный snapshot. + ## Контекст `v8-runner` должен ускорять повторные build/load сценарии, но не имеет права выполнять неполную загрузку, если нет уверенности в корректности набора изменённых файлов. @@ -23,9 +27,9 @@ Правила: 1. Анализ изменений запускается только во время команды, которой нужен build/export/load decision; background watcher не используется. -2. Persistent state хранится под `workPath/hash-storages` в отдельном `redb` context на логический source-set context. -3. Для `format=DESIGNER` используется один context на `source-set`: `designer-`. -4. Для `format=EDT` используется два context на `source-set`: `edt-` для решения об export и `designer-` для решения о load generated Designer output. +2. Persistent state хранится под `workPath/ib-state/v1//-/hash-storage.redb`; legacy `workPath/hash-storages` не мигрируется и не переиспользуется. +3. Для `format=DESIGNER` используется один логический Designer context на `source-set`, изолированный по ИБ и source identity. +4. Для `format=EDT` используются отдельные EDT-source и generated-Designer contexts, также изолированные по ИБ и source identity. 5. Scanner использует watermark/mtime filter с coarse margin и проверкой хеша для candidate files. 6. Scanner игнорирует runtime/build каталоги и файлы, которые не должны участвовать в source snapshot, например `.git`, `build`, `target`, `temp`, `tmp`, `.yaxunit`, `ConfigDumpInfo.xml`. 7. Recoverable scan/storage ошибки приводят к safe fallback: full execution или full rescan вместо partial decision. @@ -36,6 +40,7 @@ 12. Изменения `.bsl` расширяются только до существующих связанных XML-файлов, включая sibling XML и ancestor XML descriptors; каталоги в `-listFile` не добавляются, потому что Designer partial load должен получать file-only список. 13. Designer partial-load `listFile` пишется как UTF-8 с ровно одним BOM `EF BB BF` перед payload; записи остаются относительными к source-set root, используют нативные разделители компонентов пути, разделяются `CRLF` без завершающего `CRLF`, а пустой payload представлен BOM-only файлом. Этот byte contract относится только к Designer `/LoadConfigFromFiles -partial -listFile` и не меняет IBCMD partial import или partial dump list files. 14. Prepared snapshot коммитится только после успешного соответствующего export/load step. +15. Отсутствие scoped state означает `Bootstrap` и full operation даже для пустого source-set; оно никогда не интерпретируется как `NoChanges`. ## Неграницы (Non-goals) @@ -49,7 +54,7 @@ 1. Повторные build/load команды могут пропускать неизменённые source-set или выполнять partial load. 2. При сомнениях система должна выбирать full execution, а не потенциально неполную загрузку. -3. `source-set.name` и layout `workPath/hash-storages` являются частью runtime contract. +3. Versioned layout `workPath/ib-state/v1` и входы per-IB/per-source identity являются частью runtime contract; fingerprints остаются opaque. 4. Изменения ignored paths, context naming или partial decision rules являются архитектурно значимыми и требуют обновления этого ADR или нового ADR. 5. EDT flow обязан анализировать и коммитить EDT export context отдельно от Designer load context. @@ -65,10 +70,10 @@ 6. `src/use_cases/build_project.rs` применяет analysis перед platform operations и коммитит snapshots после успешного шага. 7. Для `format=EDT` build pipeline разделён на независимые последовательные стадии: - `edt-*` analysis управляет только export decision; - - successful export коммитит `edt-*` snapshot; + - successful export подготавливает EDT observation, не публикуя её отдельно; - `designer-*` analysis запускается всегда после успешной или skipped EDT stage; - Designer/IBCMD load/apply запускается только при изменениях в generated Designer output; - - `designer-*` snapshot коммитится только после successful load/apply; + - EDT observation и Designer state коммитятся согласованной generation только после successful load/apply; - ошибка на предыдущей стадии останавливает pipeline до следующей стадии. При дальнейших изменениях: @@ -76,18 +81,18 @@ 1. новые build/export flows должны использовать `ChangeAnalysis` вместо самостоятельного обхода файлов; 2. новые partial load rules должны покрываться unit tests в `change_detection::partial_load`; 3. изменения byte contract для Designer partial-load `listFile` должны покрываться точными byte-level tests, включая BOM, UTF-8 имена, `CRLF` и пустой список; -4. любые изменения context naming или storage layout должны обновлять ADR-0002 и этот ADR; +4. любые изменения runtime identity или storage layout должны обновлять ADR-0002, ADR-0023 и этот ADR; 5. failure handling должен сохранять safe fallback semantics для recoverable ошибок. ## Верификация - [x] ADR фиксирует on-demand, а не watcher-based change detection. -- [x] ADR фиксирует per-context `redb` storage под `workPath/hash-storages`. +- [x] ADR фиксирует per-IB/per-source `redb` storage под `workPath/ib-state/v1` и deliberate non-migration. - [x] ADR фиксирует разные contexts для EDT source и generated Designer output. - [x] ADR фиксирует conservative full-load fallback для unsafe partial cases. - [x] ADR фиксирует Designer partial-load `listFile` как UTF-8 with BOM и `CRLF` byte contract. - [x] ADR фиксирует commit snapshot только после successful platform step. -- [ ] EDT build выполняет Designer analysis всегда после successful или skipped EDT stage. -- [ ] Отсутствие изменений в generated Designer output приводит к skip без load/apply. -- [ ] Изменения в generated Designer output проходят через обычное partial/full decision. -- [ ] `designer-*` snapshot не коммитится до successful load/apply. +- [x] EDT build выполняет Designer analysis всегда после successful или skipped EDT stage. +- [x] Отсутствие изменений в generated Designer output приводит к skip без load/apply. +- [x] Изменения в generated Designer output проходят через обычное partial/full decision. +- [x] Согласованная state generation не коммитится до successful load/apply. diff --git a/spec/decisions/0015-atomarnaya-publikatsiya-dump-artifacts-cherez-staging-backup.md b/spec/decisions/0015-atomarnaya-publikatsiya-dump-artifacts-cherez-staging-backup.md index 6c68805..2e6fe23 100644 --- a/spec/decisions/0015-atomarnaya-publikatsiya-dump-artifacts-cherez-staging-backup.md +++ b/spec/decisions/0015-atomarnaya-publikatsiya-dump-artifacts-cherez-staging-backup.md @@ -3,6 +3,12 @@ - Статус: `accepted` - Дата: `2026-04-21` +> Уточнено [ADR-0023](0023-izolirovat-runtime-state-po-infobase-i-ispolzovat-private-shadow.md): +> sibling staging/backup остаётся действующим контрактом для package/external artifacts. +> Все режимы `dump` теперь выполняются в private shadow и публикуют только managed-file +> manifest через recoverable source journal; старые различия full и incremental/partial +> publication ниже считаются заменёнными. + ## Контекст Команды `dump` и `artifacts` публикуют результат не только во внутренний `workPath`, но и в пользовательский target path: @@ -40,23 +46,24 @@ Timeout/cancellation policy из ADR-0014 задаёт terminal-state semantics, 11. Orphan cleanup не должен удалять malformed, foreign или recent temp paths. 12. Publication phase после перемещения target в backup является filesystem critical phase; cancellation/timeout не должны hard-kill этот участок по умолчанию согласно ADR-0014. -Операции, покрытые этим контрактом: +Операции, покрытые sibling staging/backup контрактом после ADR-0023: + +1. `artifacts cf`; +2. `artifacts cfe`; +3. `artifacts epf/erf` при публикации output directory. -1. `dump --mode full` для `builder=DESIGNER`; -2. `dump --mode full` для `builder=IBCMD`; -3. `artifacts cf`; -4. `artifacts cfe`; -5. `artifacts epf/erf` при публикации output directory. +`dump --mode full|incremental|partial` использует отдельный manifest-scoped contract из +ADR-0023: platform пишет private shadow, затем B/S/D merge публикует управляемые файлы с +journal/rollback и не заменяет source root целиком. ## Неграницы (Non-goals) -1. Не обещать атомарность `dump --mode incremental`. -2. Не обещать атомарность `dump --mode partial`. -3. Не гарантировать crash-consistency на всех файловых системах, где `rename`/`fsync` не дают нужной семантики. -4. Не вводить distributed lock для target paths. -5. Не заменять workspace lock из ADR-0011. -6. Не делать имена staging/backup публичным API; публичным является safety contract, а не точный prefix. -7. Не сохранять backup после успешного publish как user-facing rollback feature. +1. Не обещать мгновенную multi-file filesystem atomicity manifest-публикации `dump` при аварии процесса; вместо неё гарантируется детерминированное restart recovery. +2. Не гарантировать crash-consistency на всех файловых системах, где `rename`/`fsync` не дают нужной семантики. +3. Не вводить distributed lock для target paths. +4. Не заменять workspace lock из ADR-0011. +5. Не делать имена staging/backup публичным API; публичным является safety contract, а не точный prefix. +6. Не сохранять backup после успешного publish как user-facing rollback feature. ## Последствия @@ -64,8 +71,8 @@ Timeout/cancellation policy из ADR-0014 задаёт terminal-state semantics, 2. Platform failure до publish сохраняет старый target. 3. Publish failure должен пытаться восстановить старый target из backup. 4. Успешная публикация может вернуть cleanup warning, если backup или metadata не удалось удалить. -5. Инкрементальные и частичные dump-режимы остаются отдельными non-atomic update modes и должны быть описаны как такие. -6. Изменения helper-ов `replace_dir_atomically`, `replace_file_atomically`, orphan cleanup или target validation требуют обновления этого ADR. +5. Все dump-режимы обязаны использовать private shadow и recoverable manifest publication из ADR-0023. +6. Изменения helper-ов `replace_dir_atomically`, `replace_file_atomically`, source journal, orphan cleanup или target validation требуют обновления связанного ADR. ## План реализации @@ -73,11 +80,10 @@ Timeout/cancellation policy из ADR-0014 задаёт terminal-state semantics, 1. `src/support/fs.rs` содержит `replace_dir_atomically` и `replace_file_atomically`. 2. `src/support/fs.rs` содержит `TempDirMetadata`, `TempDirKind`, metadata sidecars и best-effort parent fsync. -3. `src/use_cases/dump_config.rs` выполняет full Designer dump через `.dump-stage-*` и `replace_dir_atomically`. -4. `src/use_cases/dump_config.rs` выполняет full IBCMD dump через staging directory и `replace_dir_atomically`. -5. `src/use_cases/artifacts.rs` выполняет CF/CFE export через staging file и `replace_file_atomically`. -6. `src/use_cases/artifacts.rs` выполняет EPF/ERF publication через staging directory и `replace_dir_atomically`. -7. `src/use_cases/dump_config.rs` и `src/use_cases/artifacts.rs` используют target-specific advisory locks and target identity for stale cleanup. +3. `src/use_cases/dump_config.rs` выполняет любой dump в private shadow и передаёт manifest в `source_publication`. +4. `src/use_cases/artifacts.rs` выполняет CF/CFE export через staging file и `replace_file_atomically`. +5. `src/use_cases/artifacts.rs` выполняет EPF/ERF publication через staging directory и `replace_dir_atomically`. +6. Dump и artifacts используют соответствующие locks и target identity checks. Resolved follow-up к `2026-04-23`: @@ -93,14 +99,14 @@ Resolved follow-up к `2026-04-23`: При дальнейших изменениях: 1. новые full-replacement export/publish сценарии должны использовать общий staging/backup helper; -2. direct write в target разрешён только для явно non-atomic incremental/partial update modes; +2. direct platform write в project source запрещён для всех dump-режимов; 3. tests должны проверять сохранение старого target при platform failure и rollback при publish failure; 4. orphan cleanup tests должны проверять metadata matching, TTL, foreign/malformed metadata и recent paths. ## Верификация -- [x] ADR фиксирует staging/backup publication для full replacement dump/artifacts. -- [x] ADR не обещает атомарность incremental/partial dump. +- [x] ADR фиксирует staging/backup publication для full-replacement artifacts и отдельный ADR-0023 contract для dump. +- [x] ADR не обещает мгновенную multi-file filesystem atomicity dump, но требует recoverable manifest publication. - [x] ADR фиксирует rollback semantics при publish failure. - [x] ADR фиксирует cleanup warning как degraded success, а не failed publish. - [x] ADR связывает publication phase с critical filesystem mutation из ADR-0014. diff --git a/spec/decisions/0023-izolirovat-runtime-state-po-infobase-i-ispolzovat-private-shadow.md b/spec/decisions/0023-izolirovat-runtime-state-po-infobase-i-ispolzovat-private-shadow.md new file mode 100644 index 0000000..c0a52d1 --- /dev/null +++ b/spec/decisions/0023-izolirovat-runtime-state-po-infobase-i-ispolzovat-private-shadow.md @@ -0,0 +1,210 @@ +# ADR-0023: Изолировать runtime state по информационной базе и использовать private shadow + +- Статус: `accepted` +- Дата: `2026-07-21` +- Связанная задача: [#30](https://github.com/alkoleft/v8-runner-rust/issues/30) +- Уточняет: ADR-0002, ADR-0012, ADR-0015 + +## Контекст + +Текущий change-detection context определяется только парой `designer|edt` и именем +`source-set`. Поэтому две разные информационные базы, собранные из одного checkout, +разделяют один snapshot и вторая ИБ может получить ложный `Skipped`. + +Designer load всегда передает `-updateConfigDumpInfo`, а путь загрузки указывает на +живое дерево исходников. Платформа тем самым может создать или изменить +`ConfigDumpInfo.xml` в source tree. Incremental/partial dump также пишет прямо в +исходники и не способен безопасно отличить изменения пользователя от изменений ИБ. + +## Решение + +### Версионированная runtime identity + +Runtime state хранится только под: + +```text +workPath/ib-state/v1//-/ + hash-storage.redb + ConfigDumpInfo.xml + generations// + ib-baseline// + transactions/ + runtime-state.lock +``` + +`SourceObservation` логически входит в ту же generation, но физически хранится в +`hash-storage.redb`; отдельный `source-observation/` каталог зарезервирован типовой моделью и +сейчас не материализуется. + +`infobase-fingerprint` — SHA-256 от нормализованной, не содержащей секретов identity. +Ее строит один fallible typed normalizer, а тип identity реализует `Debug` только через +готовый fingerprint: + +1. для файловой ИБ — от канонического пути ИБ; +2. для серверной ИБ — от отсортированных case-normalized параметров сервера и ссылки; +3. для IBCMD — дополнительно от `dbms.kind`, `dbms.server`, `dbms.name`; +4. plain semicolon form, raw `/F`/`-F`, `/S`/`-S` и + `/IBConnectionString` приводятся к одной tagged model; +5. `Usr`/`Pwd`, `/N`/`-N`, `/P`/`-P`, config credentials и DB credentials + исключаются до хеширования; отсутствующее значение после address/auth flag является ошибкой; +6. неизвестная raw connection form отклоняется, а не хешируется как потенциальный secret. + +Пути нормализуются через nearest-existing canonical ancestor с проверенным lexical +suffix. Это дает одинаковую identity до и после создания отсутствующего target. +Symlink в уже существующей части разрешается; появление нового symlink в suffix считается +осознанной сменой target и приводит к новой identity. + +`context-fingerprint` — SHA-256 от raw identity source-set, канонического source root, +`purpose`, `format`, backend и logical context kind (`edt` или `designer`). Поля +хешируются в versioned length-prefixed representation, чтобы исключить неоднозначную +конкатенацию. В логах fingerprints допустимы, исходные секреты — нет. + +Старый `workPath/hash-storages` не мигрируется и не переиспользуется: в нем нет +достаточной информации для надежной привязки к ИБ. Первое обращение к новому layout +является bootstrap и обязательно выполняет full operation, даже для пустого source-set. + +### Private CDFI для build + +Designer никогда не получает живое source tree как load directory: + +1. исходники копируются в private transaction directory с сохранением relative paths; +2. symlink не обходятся, `ConfigDumpInfo.xml` и вложенный `workPath` исключаются; +3. валидный private `ConfigDumpInfo.xml` добавляется в staging; source CDFI игнорируется; +4. missing/corrupt private CDFI переводит план в full bootstrap без синтеза UUID/version; +5. partial list строится относительно оригинального root и применим к идентичным путям staging; +6. после успешных load и apply созданный платформой CDFI валидируется и stage-ится; + CDFI, baseline и hash snapshot становятся видимыми одним recoverable generation commit; +7. ошибка или отмена удаляет staging и не меняет source tree и успешный baseline. + +Валидация CDFI проверяет только well-formed XML, ожидаемый local root name и наличие +непустых platform-owned identity/version values, которые реально присутствуют в файле. +Она не объявляет файл совместимым с конкретной версией платформы. Ошибка platform load +с seed CDFI не ретраится автоматически как full: без надежного diagnostic code такой +retry способен скрыть ошибку исходников. Старый CDFI сохраняется, а явный full rebuild +создает transaction без seed и после успеха заменяет private CDFI. UUID/version никогда +не синтезируются и не переписываются runner-ом. + +### Private shadow и трехсторонняя публикация dump + +Full, incremental и partial dump выполняются только в private shadow. При отсутствии +baseline incremental/partial request повышается до full shadow dump. Для каждого файла +отсутствующий `B` безопасно разрешает создание нового target (`S` отсутствует) или +convergence (`S == D`). Комбинация `B=absent, S=present, D=absent` является конфликтом: +runner не удаляет локальный файл, которым ещё не владел baseline. + +Для обратной публикации сравниваются последний успешный baseline `B`, текущий source +`S` и новый shadow `D` по каждому файлу: + +- `S == B && D != B` — публиковать `D`; +- `D == B && S != B` — сохранить локальный `S`; +- `S == D` — уже согласовано; +- `S != B && D != B && S != D` — conflict; +- остальные комбинации — no-op. + +При любом conflict весь source-set остается неизменным. Перед publication source +повторно хешируется для защиты от TOCTOU. `ConfigDumpInfo.xml` никогда не публикуется +в source tree. + +Publication работает по manifest только управляемых файлов, а не заменяет source root: + +1. symlink, nested `workPath`, `.git` и прочие ignored/unmanaged entries не обходятся и + не входят в manifest; +2. до первой записи создаются byte-exact backups затронутых target и fsync journal с + generation, expected hashes и ordered actions; +3. при любой ошибке journal откатывается; незавершенный journal восстанавливается до + новой операции; +4. commit marker пишется только после всех actions и повторной проверки; cleanup + выполняется отдельно и идемпотентно. + +Source-publication journal и private state commit связывает один канонический UUID +`DumpTransactionId`. Restart recovery считает state видимым и может завершить publication +вперёд только при точном совпадении пары `(generation, transaction id)`. Совпавший номер +generation с другим token не доказывает успех этой dump-транзакции и приводит к rollback +управляемых source changes. + +Это recoverable source-set transaction: процессный сбой не оставляет неопределимое +состояние, а следующий запуск обязан завершить rollback до анализа. Она не заявляет +недостижимую multi-file filesystem atomicity в момент аварийного завершения процесса. + +### Точная квитанция + +Каждый `BuildStep` и единственный `DumpResult` содержит детерминированную, +отсортированную квитанцию следующей JSON-формы: + +```json +{ + "status": "applied|skipped|failed|conflict", + "requested": [{ "path": "...", "preHash": null, "postHash": "..." }], + "processed": [], + "skipped": [], + "conflicted": [] +} +``` + +Поля структуры private; exhaustive smart constructors не позволяют создать +противоречивые комбинации. `failed`/cancelled и `conflict` всегда имеют пустой +`processed`; `applied` не имеет `conflicted`; `skipped` не имеет `processed`. + +Эти списки являются независимыми audit dimensions, а не строгим partition. Для +`applied` одна и та же запись с одинаковыми hashes может одновременно находиться в +`processed` и `skipped`, если файл входил в effective platform scope, но B/S/D merge сохранил +локальную версию или обнаружил no-op. Для full effective scope включает полный managed D и +baseline deletions; для incremental/partial `processed` строится по наблюдаемым записям private +shadow. Другие overlap и несовпадающие hashes отклоняются. + +Списки имеют смысл: + +- `requested` — исходно обнаруженная пользовательская дельта; +- `processed` — точный effective scope: полный managed result для full либо наблюдаемый write-set + private shadow для incremental/partial, включая неизменные переписанные файлы; +- `skipped` — файлы, намеренно сохраненные/no-op; +- `conflicted` — файлы, заблокировавшие публикацию. + +Каждая запись содержит нормализованный relative path и raw SHA-256 `preHash` / +`postHash`, когда соответствующая версия существует. Для build `preHash` — last +successfully applied source observation, `postHash` — requested/staged content. Для dump +`preHash` — source перед операцией, `postHash` — proposed shadow content; поэтому conflict +также наблюдаем без публикации. Add имеет `preHash=null`, delete — `postHash=null`, а +неизменный файл partial closure — одинаковые hashes. Failed/cancelled operation хранит +дельту только в `requested`, если она уже была вычислена, не объявляет файлы processed и не +продвигает state; при более раннем failure все списки могут быть пустыми. + +### Два независимых состояния и их commit + +`IbBaseline` — полное private зеркало последнего успешно наблюдавшегося результата ИБ, +а `SourceObservation` (`redb`) — содержимое, которое последний раз было успешно +применено к ИБ. Они представлены разными типами и одной state generation. + +После dump applied/converged paths продвигаются к `D`; retained-local paths сохраняют +прежние baseline/observation `B`, чтобы следующий build снова увидел локальную дельту; +при conflict не продвигается ничего. CDFI, baseline manifest/files и redb snapshot +готовятся в private state transaction и коммитятся с journal/recovery generation. +Безопасный порядок делает новое состояние видимым только после source publication; +после state commit source journal помечается видимым точной парой `(generation, +DumpTransactionId)`. Незавершенный state journal восстанавливается до следующего planning pass. + +## Последствия + +1. Смена ИБ, source root, формата или backend создает независимый state и full bootstrap. +2. Две ИБ в одном checkout не могут влиять на skip/partial decision друг друга. +3. Clean clone не требует `ConfigDumpInfo.xml`; private CDFI появляется только как + результат успешной операции платформы. +4. Runtime layout становится диагностическим контрактом, но fingerprints являются + opaque: пользователи не должны вычислять или редактировать их вручную. +5. Копирование в private staging увеличивает локальный I/O, но делает build/dump + транзакционными относительно пользовательских исходников. +6. Вложенный `workPath` должен исключаться не по имени каталога, а по разрешенному root. + +## Верификация + +- [x] A build -> repeat skip; B build -> full; return to A -> skip; restart сохраняет результат. +- [x] Credentials не влияют на fingerprint и не встречаются в диагностике. +- [x] Missing/corrupt private CDFI всегда дает full bootstrap. +- [x] Designer build failure не создает и не изменяет source `ConfigDumpInfo.xml`. +- [x] Build receipt различает requested/processed/skipped/conflicted с raw hashes и допустимый processed/skipped overlap. +- [ ] Отдельные orchestration-тесты incremental/partial dump подтверждают, что conflict не изменяет ни одного source-файла; общая B/S/D и manifest-механика уже покрыта unit-тестами. +- [x] Scanner и private copy исключают CDFI, symlinks и вложенный custom `workPath`. +- [x] Crash recovery требует exact transaction token и не затрагивает unmanaged entries. +- [ ] `windows-latest` contract CI подтверждает Windows-specific claim/replace/remove semantics; локальная Unix-проверка не заменяет этот gate. +- [ ] Disposable real file-IB acceptance подтверждает lifecycle private CDFI; без нее PR + не объявляет задачу закрытой. diff --git a/spec/decisions/README.md b/spec/decisions/README.md index 9575fe4..a6aec55 100644 --- a/spec/decisions/README.md +++ b/spec/decisions/README.md @@ -18,7 +18,7 @@ - [ADR-0012: On-demand change detection и файловая partial-load стратегия](0012-on-demand-change-detection-i-faylovaya-partial-load-strategiya.md) — `accepted`, `2026-04-20` - [ADR-0013: MCP execution admission, timeout/cancellation routing и HTTP session capacity](0013-mcp-execution-admission-timeout-cancellation-routing-i-http-session-capacity.md) — `accepted`, `2026-04-20` - [ADR-0014: Единая timeout/cancellation policy для CLI и MCP команд](0014-edinaya-timeout-cancellation-policy-dlya-cli-i-mcp-komand.md) — `accepted`, `2026-04-20` -- [ADR-0015: Атомарная публикация dump/artifacts через staging/backup](0015-atomarnaya-publikatsiya-dump-artifacts-cherez-staging-backup.md) — `accepted`, `2026-04-21` +- [ADR-0015: Атомарная публикация dump/artifacts через staging/backup](0015-atomarnaya-publikatsiya-dump-artifacts-cherez-staging-backup.md) — `accepted`, `2026-04-21`; staging/backup для artifacts, dump contract заменён ADR-0023 - [ADR-0016: Единый `ExecutionOutcome` и pipeline steps для runner-like сценариев](0016-edinyy-executionoutcome-i-pipeline-steps-dlya-runner-like-stsenariev.md) — `accepted`, `2026-04-21` - [ADR-0017: `v8project.yaml` / `source-set` как главный конфигурационный контракт](0017-v8project-yaml-source-set-kak-glavnyy-konfiguratsionnyy-kontrakt.md) — `accepted`, `2026-04-20` - [ADR-0018: Перенести контракт информационной базы в `infobase`](0018-perenesti-kontrakt-informatsionnoy-bazy-v-infobase.md) — `accepted`, `2026-04-21` @@ -26,6 +26,7 @@ - [ADR-0020: Упростить CLI-only `convert` до repo-aware конвертации текущих исходников проекта](0020-dobavit-cli-only-convert-dlya-dvustoronney-konvertatsii-edt-i-designer.md) — `accepted`, `2026-04-22` - [ADR-0021: Ввести локальный overlay для `v8project.yaml`](0021-lokalnyy-overlay-config.md) — `accepted`, `2026-05-02` - [ADR-0022: Ввести общий механизм подготовки расширений и использовать его для `client_mcp`](0022-universalnyy-mehanizm-podgotovki-rasshireniy-i-client-mcp-extension.md) — `accepted`, `2026-05-02` +- [ADR-0023: Изолировать runtime state по информационной базе и использовать private shadow](0023-izolirovat-runtime-state-po-infobase-i-ispolzovat-private-shadow.md) — `accepted`, `2026-07-21`; `ib-state/v1`, private build/dump shadows и recoverable manifest publication ## Правила обновления diff --git a/src/change_detection/analyzer.rs b/src/change_detection/analyzer.rs index be2f047..b49b1e6 100644 --- a/src/change_detection/analyzer.rs +++ b/src/change_detection/analyzer.rs @@ -3,15 +3,21 @@ use std::path::{Path, PathBuf}; use thiserror::Error; -use crate::change_detection::hash_storage::{HashStorage, StorageError, StoredFileState}; +use crate::change_detection::hash_storage::{ + HashStorage, HashStorageLoad, ObservedHashStorage, ObservedStorageState, StorageError, + StoredFileState, +}; use crate::change_detection::scanner::{self, ScanError}; use crate::domain::source_set::SourceSetContext; /// A single detected file change. -#[derive(Debug, Clone)] +#[derive(Debug, Clone, PartialEq, Eq)] pub struct FileChange { pub path: PathBuf, + pub rel_path: String, pub kind: ChangeKind, + pub pre_hash: Option, + pub post_hash: Option, } /// How a file changed relative to the stored state. @@ -35,12 +41,22 @@ pub struct PreparedFileState { pub struct PreparedStateUpdate { pub snapshot: Vec, pub scan_started_at: u64, - pub observed_generation: u64, + pub observed_storage: ObservedStorageState, +} + +/// Deterministic current managed inventory plus its delta against persisted observation. +#[derive(Debug, Clone)] +pub struct ManagedInventory { + pub requested: Vec, + pub current: Vec, + pub prepared: PreparedStateUpdate, } /// Result of analyzing one source-set against its persisted snapshot. #[derive(Debug, Clone)] pub enum AnalysisOutcome { + /// No scoped state exists yet; callers must perform a full bootstrap. + Bootstrap, NoChanges, Changes { changes: Vec, @@ -59,6 +75,9 @@ pub struct ContextAnalysis { /// Hard failures that prevent normal change-detection flow. #[derive(Debug, Clone, Error)] pub enum ChangeDetectionError { + #[error("invalid managed inventory for source-set '{source_set}': {reason}")] + InvalidInventory { source_set: String, reason: String }, + #[error("hard storage error for source-set '{source_set}' at '{storage_path}': {reason}")] StorageHard { source_set: String, @@ -66,20 +85,26 @@ pub enum ChangeDetectionError { reason: String, }, - #[error("concurrent state modification for source-set '{source_set}' at '{storage_path}': expected generation {expected}, found {actual}")] + #[error("concurrent state modification for source-set '{source_set}' at '{storage_path}': expected generation {expected}, found {actual:?}")] ConcurrentStateModified { source_set: String, storage_path: PathBuf, expected: u64, - actual: u64, + actual: Option, }, } /// Analyze one source-set context and produce either concrete changes or a safe fallback. -pub fn analyze_context(context: &SourceSetContext, work_path: &Path) -> ContextAnalysis { - let storage = HashStorage::new(context.storage_path(work_path)); - let snapshot = match storage.load_snapshot() { - Ok(snapshot) => snapshot, +pub fn analyze_context(context: &SourceSetContext) -> ContextAnalysis { + let storage = HashStorage::new(context.storage_path()); + let snapshot = match storage.load_state() { + Ok(HashStorageLoad::MissingPath | HashStorageLoad::ExistingUninitialized) => { + return ContextAnalysis { + context: context.clone(), + outcome: Ok(AnalysisOutcome::Bootstrap), + } + } + Ok(HashStorageLoad::Initialized(snapshot)) => snapshot, Err(e) => { if e.is_recoverable() { tracing::warn!( @@ -100,7 +125,12 @@ pub fn analyze_context(context: &SourceSetContext, work_path: &Path) -> ContextA }; let stored_keys: HashSet = snapshot.entries.keys().cloned().collect(); - let scan = match scanner::scan(context.path(), snapshot.watermark, &stored_keys) { + let scan = match scanner::scan( + context.path(), + snapshot.watermark, + &stored_keys, + context.excluded_roots(), + ) { Ok(scan) => scan, Err(e) => { tracing::warn!( @@ -115,21 +145,16 @@ pub fn analyze_context(context: &SourceSetContext, work_path: &Path) -> ContextA } }; - let mut changes = detect_changes(&scan.candidates, &snapshot.entries); let seen_rel: HashSet<&str> = scan .seen_files .iter() .map(|f| f.rel_path.as_str()) .collect(); - changes.extend( - snapshot - .entries - .iter() - .filter(|(rel, _)| !seen_rel.contains(rel.as_str())) - .map(|(rel, _)| FileChange { - path: context.path().join(rel), - kind: ChangeKind::Deleted, - }), + let changes = detect_changes( + context.path(), + &scan.candidates, + &snapshot.entries, + &seen_rel, ); let prepared = build_prepared_state(&scan, &snapshot.entries, snapshot.generation); @@ -146,42 +171,109 @@ pub fn analyze_context(context: &SourceSetContext, work_path: &Path) -> ContextA } /// Analyze multiple source-set contexts using the same work directory. -pub fn analyze_contexts(contexts: &[SourceSetContext], work_path: &Path) -> Vec { - contexts +pub fn analyze_contexts(contexts: &[SourceSetContext]) -> Vec { + contexts.iter().map(analyze_context).collect() +} + +pub fn managed_inventory( + context: &SourceSetContext, +) -> Result { + let storage = HashStorage::new(context.storage_path()); + let (stored, observed_storage) = match storage + .observe_state() + .map_err(|error| map_storage_hard(context, storage.path(), error))? + { + ObservedHashStorage::MissingPath => (HashMap::new(), ObservedStorageState::MissingPath), + ObservedHashStorage::ExistingUninitialized(observation) + | ObservedHashStorage::Recoverable(observation) => (HashMap::new(), observation), + ObservedHashStorage::Initialized(snapshot) => { + let generation = snapshot.generation; + ( + snapshot.entries, + ObservedStorageState::Initialized { generation }, + ) + } + }; + let scan = scanner::scan( + context.path(), + None, + &HashSet::new(), + context.excluded_roots(), + ) + .map_err(|error| map_scan_error(context, error))?; + let seen = scan + .seen_files .iter() - .map(|ctx| analyze_context(ctx, work_path)) - .collect() + .map(|file| file.rel_path.as_str()) + .collect::>(); + let requested = detect_changes(context.path(), &scan.candidates, &stored, &seen); + let mut current = scan + .candidates + .iter() + .map(|candidate| PreparedFileState { + rel_path: candidate.rel_path.clone(), + mtime_ns: candidate.mtime_ns, + hash: candidate.hash.clone(), + }) + .collect::>(); + current.sort_by(|left, right| left.rel_path.cmp(&right.rel_path)); + let prepared = PreparedStateUpdate { + snapshot: current.clone(), + scan_started_at: scan.scan_started_at, + observed_storage, + }; + Ok(ManagedInventory { + requested, + current, + prepared, + }) +} + +/// Commit exactly the full inventory observed before the platform operation. +pub fn commit_full_observation( + context: &SourceSetContext, + prepared: &PreparedStateUpdate, +) -> Result<(), ChangeDetectionError> { + let storage = HashStorage::new(context.storage_path()); + let snapshot = to_storage_snapshot(&prepared.snapshot); + storage + .commit_observed_snapshot( + &snapshot, + prepared.scan_started_at, + &prepared.observed_storage, + ) + .map_err(|error| map_commit_error(context, storage.path(), error)) } /// Persist a prepared snapshot after the corresponding build/load step succeeded. pub fn commit_success( context: &SourceSetContext, - work_path: &Path, prepared: &PreparedStateUpdate, ) -> Result<(), ChangeDetectionError> { - let storage = HashStorage::new(context.storage_path(work_path)); + let storage = HashStorage::new(context.storage_path()); let snapshot = to_storage_snapshot(&prepared.snapshot); storage .commit_snapshot( &snapshot, prepared.scan_started_at, - prepared.observed_generation, + prepared.observed_storage.generation(), ) .map_err(|e| map_commit_error(context, storage.path(), e)) } /// Re-scan the source-set from scratch and replace the stored snapshot. -pub fn rescan_and_commit_full( - context: &SourceSetContext, - work_path: &Path, -) -> Result<(), ChangeDetectionError> { - let storage = HashStorage::new(context.storage_path(work_path)); +#[cfg(test)] +pub fn rescan_and_commit_full(context: &SourceSetContext) -> Result<(), ChangeDetectionError> { + let storage = HashStorage::new(context.storage_path()); let current_generation = match storage.current_generation() { Ok(generation) => generation, Err(e) if e.is_recoverable() => { let full = full_snapshot(context, &StorageSnapshotInputs::empty())?; + let observed = storage + .recoverable_observation() + .map_err(|err| map_commit_error(context, storage.path(), err))?; return storage - .recover_and_commit_snapshot(&full.snapshot, full.scan_started_at) + .commit_observed_snapshot(&full.snapshot, full.scan_started_at, &observed) .map_err(|err| map_commit_error(context, storage.path(), err)); } Err(e) => return Err(map_storage_hard(context, storage.path(), e)), @@ -205,23 +297,44 @@ pub fn rescan_and_commit_full( } fn detect_changes( + source_root: &Path, candidates: &[scanner::CandidateFile], stored: &HashMap, + seen_rel: &HashSet<&str>, ) -> Vec { - candidates + let mut changes = candidates .iter() .filter_map(|candidate| { - let kind = match stored.get(&candidate.rel_path) { - None => ChangeKind::Added, - Some(existing) if existing.hash != candidate.hash => ChangeKind::Modified, + let (kind, pre_hash) = match stored.get(&candidate.rel_path) { + None => (ChangeKind::Added, None), + Some(existing) if existing.hash != candidate.hash => { + (ChangeKind::Modified, Some(existing.hash.clone())) + } Some(_) => return None, }; Some(FileChange { path: candidate.path.clone(), + rel_path: candidate.rel_path.clone(), kind, + pre_hash, + post_hash: Some(candidate.hash.clone()), }) }) - .collect() + .collect::>(); + changes.extend( + stored + .iter() + .filter(|(rel_path, _)| !seen_rel.contains(rel_path.as_str())) + .map(|(rel_path, state)| FileChange { + path: source_root.join(rel_path), + rel_path: rel_path.clone(), + kind: ChangeKind::Deleted, + pre_hash: Some(state.hash.clone()), + post_hash: None, + }), + ); + changes.sort_by(|left, right| left.rel_path.cmp(&right.rel_path)); + changes } fn build_prepared_state( @@ -268,26 +381,32 @@ fn build_prepared_state( // Remove invalid placeholders introduced by missing stored state. merged.retain(|_, state| !state.hash.is_empty()); + let mut snapshot = merged + .into_iter() + .map(|(rel_path, state)| PreparedFileState { + rel_path, + mtime_ns: state.mtime_ns, + hash: state.hash, + }) + .collect::>(); + snapshot.sort_by(|left, right| left.rel_path.cmp(&right.rel_path)); PreparedStateUpdate { - snapshot: merged - .into_iter() - .map(|(rel_path, state)| PreparedFileState { - rel_path, - mtime_ns: state.mtime_ns, - hash: state.hash, - }) - .collect(), + snapshot, scan_started_at: scan.scan_started_at, - observed_generation, + observed_storage: ObservedStorageState::Initialized { + generation: observed_generation, + }, } } +#[cfg(test)] struct StorageSnapshotInputs { watermark: Option, stored_keys: HashSet, observed_generation: u64, } +#[cfg(test)] impl StorageSnapshotInputs { fn empty() -> Self { Self { @@ -298,18 +417,25 @@ impl StorageSnapshotInputs { } } +#[cfg(test)] struct FullSnapshot { snapshot: HashMap, scan_started_at: u64, observed_generation: u64, } +#[cfg(test)] fn full_snapshot( context: &SourceSetContext, input: &StorageSnapshotInputs, ) -> Result { - let scan = scanner::scan(context.path(), input.watermark, &input.stored_keys) - .map_err(|e| map_scan_error(context, e))?; + let scan = scanner::scan( + context.path(), + input.watermark, + &input.stored_keys, + context.excluded_roots(), + ) + .map_err(|e| map_scan_error(context, e))?; let mut snapshot = HashMap::new(); for candidate in scan.candidates { snapshot.insert( @@ -382,11 +508,185 @@ fn map_scan_error(context: &SourceSetContext, err: ScanError) -> ChangeDetection #[cfg(test)] mod tests { - use super::{rescan_and_commit_full, ChangeDetectionError, ChangeKind, FileChange}; + use super::{ + analyze_context, commit_full_observation, detect_changes, managed_inventory, + rescan_and_commit_full, AnalysisOutcome, ChangeDetectionError, ChangeKind, FileChange, + }; + use crate::change_detection::hash_storage::{StoredFileState, META, META_KEY_GENERATION}; use crate::change_detection::partial_load::decide; + use crate::change_detection::scanner::CandidateFile; + use crate::config::model::{BuilderBackend, InfobaseConfig, SourceFormat, SourceSetPurpose}; + use crate::domain::runtime_state::{ + InfobaseIdentity, LogicalSourceRole, RuntimeSourceDescriptor, RuntimeSourceIdentityInputs, + RuntimeStateLayout, + }; use crate::domain::source_set::SourceSetContext; + use std::collections::{HashMap, HashSet}; + use std::path::Path; use tempfile::tempdir; + fn test_context(state_root: &Path, source_root: &Path) -> SourceSetContext { + let identity = InfobaseIdentity::normalize(&InfobaseConfig::file(format!( + "File={}", + state_root.join("ib").display() + ))) + .expect("identity"); + let layout = RuntimeStateLayout::new(state_root.join("work"), identity).expect("layout"); + let descriptor = RuntimeSourceDescriptor::new(RuntimeSourceIdentityInputs { + configured_source_identity: Path::new("src"), + source_root, + purpose: SourceSetPurpose::Configuration, + format: SourceFormat::Designer, + backend: BuilderBackend::Designer, + logical_role: LogicalSourceRole::DesignerSource, + }) + .expect("descriptor"); + SourceSetContext::new( + "main", + source_root.to_path_buf(), + layout.source_state("main", &descriptor), + ) + } + + #[test] + fn file_changes_reuse_candidate_and_stored_hashes_for_all_delta_kinds() { + let root = Path::new("/source"); + let candidates = vec![ + CandidateFile { + path: root.join("added.xml"), + rel_path: "added.xml".to_owned(), + mtime_ns: 1, + hash: "added-post".to_owned(), + }, + CandidateFile { + path: root.join("modified.xml"), + rel_path: "modified.xml".to_owned(), + mtime_ns: 2, + hash: "modified-post".to_owned(), + }, + ]; + let stored = HashMap::from([ + ( + "modified.xml".to_owned(), + StoredFileState { + mtime_ns: 1, + hash: "modified-pre".to_owned(), + }, + ), + ( + "deleted.xml".to_owned(), + StoredFileState { + mtime_ns: 1, + hash: "deleted-pre".to_owned(), + }, + ), + ]); + let seen = HashSet::from(["added.xml", "modified.xml"]); + + let changes = detect_changes(root, &candidates, &stored, &seen); + + assert_eq!( + changes + .iter() + .map(|change| ( + change.rel_path.as_str(), + change.kind.clone(), + change.pre_hash.as_deref(), + change.post_hash.as_deref(), + )) + .collect::>(), + vec![ + ("added.xml", ChangeKind::Added, None, Some("added-post")), + ( + "deleted.xml", + ChangeKind::Deleted, + Some("deleted-pre"), + None, + ), + ( + "modified.xml", + ChangeKind::Modified, + Some("modified-pre"), + Some("modified-post"), + ), + ] + ); + } + + #[test] + fn missing_scoped_storage_requires_bootstrap_even_for_empty_source() { + let dir = tempdir().expect("tempdir"); + let source_root = dir.path().join("src"); + std::fs::create_dir(&source_root).expect("source"); + let context = test_context(dir.path(), &source_root); + + let analysis = analyze_context(&context); + + assert!(matches!(analysis.outcome, Ok(AnalysisOutcome::Bootstrap))); + assert!(!context.storage_path().exists()); + } + + #[test] + fn existing_empty_scoped_database_requires_bootstrap() { + let dir = tempdir().expect("tempdir"); + let source_root = dir.path().join("src"); + std::fs::create_dir(&source_root).expect("source"); + let context = test_context(dir.path(), &source_root); + std::fs::create_dir_all(context.storage_path().parent().expect("storage parent")) + .expect("storage parent"); + redb::Database::create(context.storage_path()).expect("empty database"); + + assert!(matches!( + analyze_context(&context).outcome, + Ok(AnalysisOutcome::Bootstrap) + )); + } + + #[test] + fn metadata_only_storage_falls_back_and_can_be_recovered() { + let dir = tempdir().expect("tempdir"); + let source_root = dir.path().join("src"); + std::fs::create_dir(&source_root).expect("source"); + std::fs::write(source_root.join("Configuration.xml"), "").expect("config"); + let context = test_context(dir.path(), &source_root); + std::fs::create_dir_all(context.storage_path().parent().expect("storage parent")) + .expect("storage parent"); + let database = redb::Database::create(context.storage_path()).expect("database"); + let write = database.begin_write().expect("write"); + { + let mut meta = write.open_table(META).expect("meta"); + meta.insert(META_KEY_GENERATION, 1).expect("generation"); + } + write.commit().expect("commit"); + drop(database); + + assert!(matches!( + analyze_context(&context).outcome, + Ok(AnalysisOutcome::Fallback) + )); + rescan_and_commit_full(&context).expect("recover storage"); + assert!(matches!( + analyze_context(&context).outcome, + Ok(AnalysisOutcome::NoChanges) + )); + } + + #[test] + fn directory_at_scoped_storage_path_is_a_hard_error_not_bootstrap() { + let dir = tempdir().expect("tempdir"); + let source_root = dir.path().join("src"); + std::fs::create_dir(&source_root).expect("source"); + let context = test_context(dir.path(), &source_root); + std::fs::create_dir_all(context.storage_path()).expect("invalid storage directory"); + + let analysis = analyze_context(&context); + + assert!(matches!( + analysis.outcome, + Err(ChangeDetectionError::StorageHard { .. }) + )); + } + #[test] fn partial_load_contract_stays_compatible_with_file_change() { let dir = tempdir().expect("tempdir"); @@ -398,7 +698,10 @@ mod tests { let changes = vec![FileChange { path: module, + rel_path: "Catalogs.Items/ObjectModule.bsl".to_owned(), kind: ChangeKind::Modified, + pre_hash: Some("old".to_owned()), + post_hash: Some("new".to_owned()), }]; let decision = decide( &changes, @@ -415,16 +718,92 @@ mod tests { fn hard_storage_errors_stay_hard_during_full_rescan() { let dir = tempdir().expect("tempdir"); let source_root = dir.path().join("src"); - let work_path = dir.path().join("work"); std::fs::create_dir_all(&source_root).expect("source"); std::fs::write(source_root.join("Configuration.xml"), "").expect("config"); - let storage_path = work_path.join("hash-storages").join("designer-main.redb"); + let context = test_context(dir.path(), &source_root); + let storage_path = context.storage_path(); std::fs::create_dir_all(&storage_path).expect("storage dir"); + let error = rescan_and_commit_full(&context).expect_err("expected hard error"); + + assert!(matches!(error, ChangeDetectionError::StorageHard { .. })); + } + + #[test] + fn managed_inventory_reports_only_actual_delta_and_keeps_full_current_snapshot() { + let dir = tempdir().expect("tempdir"); + let source_root = dir.path().join("src"); + std::fs::create_dir_all(&source_root).expect("source"); + std::fs::write(source_root.join("Configuration.xml"), "before").expect("config"); + std::fs::write(source_root.join("Languages.xml"), "stable").expect("language"); + let context = test_context(dir.path(), &source_root); + rescan_and_commit_full(&context).expect("initialize"); + + let unchanged = managed_inventory(&context).expect("unchanged inventory"); + assert!(unchanged.requested.is_empty()); + assert_eq!(unchanged.current.len(), 2); + + std::fs::write(source_root.join("Configuration.xml"), "after").expect("modify"); + let modified = managed_inventory(&context).expect("modified inventory"); + assert_eq!(modified.requested.len(), 1); + assert_eq!(modified.requested[0].rel_path, "Configuration.xml"); + assert_eq!(modified.requested[0].kind, ChangeKind::Modified); + assert_eq!(modified.current.len(), 2); + } - let context = SourceSetContext::new("main", source_root, "designer-main"); - let error = rescan_and_commit_full(&context, &work_path).expect_err("expected hard error"); + #[test] + fn managed_inventory_propagates_hard_storage_errors() { + let dir = tempdir().expect("tempdir"); + let source_root = dir.path().join("src"); + std::fs::create_dir_all(&source_root).expect("source"); + let context = test_context(dir.path(), &source_root); + std::fs::create_dir_all(context.storage_path()).expect("invalid storage directory"); + let error = managed_inventory(&context).expect_err("hard error"); assert!(matches!(error, ChangeDetectionError::StorageHard { .. })); } + + #[test] + fn stale_full_observation_propagates_concurrent_state_modification() { + let dir = tempdir().expect("tempdir"); + let source_root = dir.path().join("src"); + std::fs::create_dir_all(&source_root).expect("source"); + std::fs::write(source_root.join("Configuration.xml"), "source").expect("config"); + let context = test_context(dir.path(), &source_root); + let stale = managed_inventory(&context).expect("observation").prepared; + rescan_and_commit_full(&context).expect("concurrent commit"); + + let error = commit_full_observation(&context, &stale).expect_err("stale commit"); + assert!(matches!( + error, + ChangeDetectionError::ConcurrentStateModified { .. } + )); + } + + #[test] + fn recoverable_storage_commits_the_prepared_full_observation() { + let dir = tempdir().expect("tempdir"); + let source_root = dir.path().join("src"); + std::fs::create_dir_all(&source_root).expect("source"); + std::fs::write(source_root.join("Configuration.xml"), "source").expect("config"); + let context = test_context(dir.path(), &source_root); + std::fs::create_dir_all(context.storage_path().parent().expect("storage parent")) + .expect("storage parent"); + let database = redb::Database::create(context.storage_path()).expect("database"); + let write = database.begin_write().expect("write"); + { + let mut meta = write.open_table(META).expect("meta"); + meta.insert(META_KEY_GENERATION, 1).expect("generation"); + } + write.commit().expect("commit"); + drop(database); + + let observation = managed_inventory(&context).expect("recoverable observation"); + assert_eq!(observation.requested.len(), 1); + commit_full_observation(&context, &observation.prepared).expect("recover commit"); + assert!(matches!( + analyze_context(&context).outcome, + Ok(AnalysisOutcome::NoChanges) + )); + } } diff --git a/src/change_detection/hash_storage.rs b/src/change_detection/hash_storage.rs index 63b4c15..b5f9064 100644 --- a/src/change_detection/hash_storage.rs +++ b/src/change_detection/hash_storage.rs @@ -1,10 +1,31 @@ use redb::{ - Database, DatabaseError, ReadableTable, StorageError as RedbStorageError, TableDefinition, - TableError, TransactionError, + CommitError, Database, DatabaseError, ReadableTable, StorageError as RedbStorageError, + TableDefinition, TableError, TransactionError, }; +use sha2::{Digest, Sha256}; use std::collections::{HashMap, HashSet}; +use std::fs::{self, OpenOptions}; +use std::io::Read; use std::path::{Path, PathBuf}; use thiserror::Error; +use uuid::Uuid; + +use crate::domain::runtime_state::DumpTransactionId; + +#[cfg(test)] +thread_local! { + static BEFORE_RECOVERY_CLAIM_HOOK: std::cell::RefCell>> = + const { std::cell::RefCell::new(None) }; + static BEFORE_MISSING_PUBLISH_HOOK: std::cell::RefCell>> = + const { std::cell::RefCell::new(None) }; + static AFTER_RECOVERY_CLAIM_HOOK: std::cell::RefCell>> = + const { std::cell::RefCell::new(None) }; + static BEFORE_RECOVERY_ROLLBACK_HOOK: std::cell::RefCell>> = + const { std::cell::RefCell::new(None) }; + static FORCE_RECOVERY_PUBLISH_FAILURE: std::cell::Cell = const { std::cell::Cell::new(false) }; + static BEFORE_EXACT_OBSERVATION_HOOK: std::cell::RefCell>> = + const { std::cell::RefCell::new(None) }; +} /// `redb` table with per-file mtimes keyed by relative path. pub const FILES_MTIME: TableDefinition<&str, u64> = TableDefinition::new("files_mtime"); @@ -16,6 +37,8 @@ pub const META: TableDefinition<&str, u64> = TableDefinition::new("meta"); pub const META_KEY_WATERMARK: &str = "watermark"; /// Metadata key storing optimistic-lock generation. pub const META_KEY_GENERATION: &str = "generation"; +const META_KEY_DUMP_TRANSACTION_HI: &str = "dump_transaction_hi"; +const META_KEY_DUMP_TRANSACTION_LO: &str = "dump_transaction_lo"; /// Persisted state for one file entry inside the storage snapshot. #[derive(Debug, Clone)] @@ -32,6 +55,62 @@ pub struct StorageSnapshot { pub generation: u64, } +/// Distinguishes a never-initialized scoped storage from a committed snapshot. +#[derive(Debug, Clone)] +pub enum HashStorageLoad { + MissingPath, + ExistingUninitialized, + Initialized(StorageSnapshot), +} + +#[derive(Debug, Clone)] +pub enum ObservedHashStorage { + MissingPath, + ExistingUninitialized(ObservedStorageState), + Initialized(StorageSnapshot), + Recoverable(ObservedStorageState), +} + +/// Exact storage observation attached to a prepared source snapshot. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum ObservedStorageState { + MissingPath, + ExistingUninitialized { + identity: StorageFileIdentity, + sha256: String, + }, + Initialized { + generation: u64, + }, + Recoverable { + generation: u64, + identity: StorageFileIdentity, + sha256: String, + }, +} + +impl ObservedStorageState { + pub fn generation(&self) -> u64 { + match self { + Self::MissingPath | Self::ExistingUninitialized { .. } => 0, + Self::Initialized { generation } | Self::Recoverable { generation, .. } => *generation, + } + } +} + +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct StorageFileIdentity { + pub len: u64, + #[cfg(unix)] + pub device: u64, + #[cfg(unix)] + pub inode: u64, + #[cfg(windows)] + pub volume_serial: u32, + #[cfg(windows)] + pub file_index: u64, +} + /// Storage-layer failures split into recoverable and hard categories. #[derive(Debug, Error)] pub enum StorageError { @@ -41,11 +120,11 @@ pub enum StorageError { #[error("hard storage problem for '{path}': {reason}")] Hard { path: PathBuf, reason: String }, - #[error("storage state changed concurrently for '{path}': expected generation {expected}, found {actual}")] + #[error("storage state changed concurrently for '{path}': expected generation {expected}, found {actual:?}")] ConcurrentStateModified { path: PathBuf, expected: u64, - actual: u64, + actual: Option, }, } @@ -73,12 +152,148 @@ impl HashStorage { &self.path } - /// Load the current snapshot from storage. - pub fn load_snapshot(&self) -> Result { - if !self.path.exists() { - return Ok(StorageSnapshot::default()); + /// Capture the exact bytes and file identity of a recoverable storage file. + pub fn recoverable_observation(&self) -> Result { + let generation = match self.recoverable_generation() { + Ok(generation) => generation, + Err(error) if error.is_recoverable() => 0, + Err(error) => return Err(error), + }; + let (identity, sha256) = self.raw_file_token()?; + let observed = ObservedStorageState::Recoverable { + generation, + identity, + sha256, + }; + self.verify_observation_class(&observed)?; + Ok(observed) + } + + pub(crate) fn uninitialized_observation(&self) -> Result { + let (identity, sha256) = self.raw_file_token()?; + let observed = ObservedStorageState::ExistingUninitialized { identity, sha256 }; + self.verify_observation_class(&observed)?; + Ok(observed) + } + + /// Classify storage and capture any exact-file token as one validated observation. + pub fn observe_state(&self) -> Result { + let classified = self.load_state(); + #[cfg(test)] + run_test_hook(&BEFORE_EXACT_OBSERVATION_HOOK); + match classified { + Ok(HashStorageLoad::MissingPath) => Ok(ObservedHashStorage::MissingPath), + Ok(HashStorageLoad::ExistingUninitialized) => self + .uninitialized_observation() + .map(ObservedHashStorage::ExistingUninitialized), + Ok(HashStorageLoad::Initialized(snapshot)) => { + Ok(ObservedHashStorage::Initialized(snapshot)) + } + Err(error) if error.is_recoverable() => self + .recoverable_observation() + .map(ObservedHashStorage::Recoverable), + Err(error) => Err(error), + } + } + + fn verify_observation_class( + &self, + expected: &ObservedStorageState, + ) -> Result<(), StorageError> { + let still_same_class = match (expected, self.load_state()) { + ( + ObservedStorageState::ExistingUninitialized { .. }, + Ok(HashStorageLoad::ExistingUninitialized), + ) => true, + (ObservedStorageState::Recoverable { .. }, Err(error)) if error.is_recoverable() => { + true + } + (_, Err(error)) if !error.is_recoverable() => return Err(error), + _ => false, + }; + if still_same_class { + Ok(()) + } else { + Err(StorageError::ConcurrentStateModified { + path: self.path.clone(), + expected: expected.generation(), + actual: self.observed_generation_for_conflict()?, + }) } - let db = Database::open(&self.path).map_err(|e| map_database_error(&self.path, e))?; + } + + fn raw_file_token(&self) -> Result<(StorageFileIdentity, String), StorageError> { + let mut options = OpenOptions::new(); + options.read(true); + #[cfg(unix)] + { + use std::os::unix::fs::OpenOptionsExt; + options.custom_flags(libc::O_NOFOLLOW | libc::O_CLOEXEC); + } + #[cfg(windows)] + { + use std::os::windows::fs::OpenOptionsExt; + const FILE_FLAG_OPEN_REPARSE_POINT: u32 = 0x0020_0000; + options.custom_flags(FILE_FLAG_OPEN_REPARSE_POINT); + } + let mut file = options + .open(&self.path) + .map_err(|error| StorageError::Hard { + path: self.path.clone(), + reason: format!("open storage without following links: {error}"), + })?; + let before = file.metadata().map_err(|error| StorageError::Hard { + path: self.path.clone(), + reason: format!("inspect opened storage: {error}"), + })?; + if !before.file_type().is_file() || storage_metadata_is_reparse_point(&before) { + return Err(StorageError::Hard { + path: self.path.clone(), + reason: "recoverable storage is not a regular file".to_owned(), + }); + } + let identity = storage_file_identity(&file, &before, &self.path)?; + let mut digest = Sha256::new(); + let mut buffer = [0_u8; 64 * 1024]; + loop { + let read = file.read(&mut buffer).map_err(|error| StorageError::Hard { + path: self.path.clone(), + reason: format!("read recoverable storage bytes: {error}"), + })?; + if read == 0 { + break; + } + digest.update(&buffer[..read]); + } + let after = file.metadata().map_err(|error| StorageError::Hard { + path: self.path.clone(), + reason: format!("re-inspect opened storage: {error}"), + })?; + if storage_file_identity(&file, &after, &self.path)? != identity { + return Err(StorageError::ConcurrentStateModified { + path: self.path.clone(), + expected: 0, + actual: None, + }); + } + Ok((identity, format!("{:x}", digest.finalize()))) + } + + /// Load typed initialization state without a separate filesystem existence check. + pub fn load_state(&self) -> Result { + match self.path.try_exists() { + Ok(false) => return Ok(HashStorageLoad::MissingPath), + Ok(true) => {} + Err(error) => return Err(map_filesystem_lookup_error(&self.path, error)), + } + let db = match Database::open(&self.path) { + Ok(database) => database, + Err(database_error) => match self.path.try_exists() { + Ok(false) => return Ok(HashStorageLoad::MissingPath), + Ok(true) => return Err(map_database_error(&self.path, database_error)), + Err(error) => return Err(map_filesystem_lookup_error(&self.path, error)), + }, + }; let tx = db .begin_read() .map_err(|e| map_tx_error(&self.path, e, "begin read"))?; @@ -103,11 +318,14 @@ impl HashStorage { let hash_exists = hash_tbl.is_some(); if !mtime_exists || !hash_exists { if !mtime_exists && !hash_exists { - return Ok(StorageSnapshot { - entries: HashMap::new(), - watermark: read_watermark(meta_tbl.as_ref(), &self.path)?, - generation: read_generation(meta_tbl.as_ref(), &self.path)?, - }); + return if meta_tbl.is_none() { + Ok(HashStorageLoad::ExistingUninitialized) + } else { + Err(StorageError::Recoverable { + path: self.path.clone(), + reason: "metadata exists without snapshot tables".to_owned(), + }) + }; } return Err(StorageError::Recoverable { path: self.path.clone(), @@ -115,7 +333,10 @@ impl HashStorage { }); } let (Some(mtime_tbl), Some(hash_tbl)) = (mtime_tbl, hash_tbl) else { - unreachable!("table presence checked above"); + return Err(StorageError::Recoverable { + path: self.path.clone(), + reason: "snapshot tables disappeared while loading storage".to_owned(), + }); }; let mut entries = HashMap::new(); @@ -158,11 +379,21 @@ impl HashStorage { } } - Ok(StorageSnapshot { + Ok(HashStorageLoad::Initialized(StorageSnapshot { entries, watermark: read_watermark(meta_tbl.as_ref(), &self.path)?, generation: read_generation(meta_tbl.as_ref(), &self.path)?, - }) + })) + } + + #[cfg(test)] + pub fn load_snapshot(&self) -> Result { + match self.load_state()? { + HashStorageLoad::MissingPath | HashStorageLoad::ExistingUninitialized => { + Ok(StorageSnapshot::default()) + } + HashStorageLoad::Initialized(snapshot) => Ok(snapshot), + } } /// Persist a full snapshot if the caller still owns the expected generation. @@ -191,7 +422,7 @@ impl HashStorage { return Err(StorageError::ConcurrentStateModified { path: self.path.clone(), expected: expected_generation, - actual: current_generation, + actual: Some(current_generation), }); } @@ -207,36 +438,368 @@ impl HashStorage { .map_err(|e| map_storage_error(&self.path, "write watermark", e))?; meta.insert(META_KEY_GENERATION, expected_generation + 1) .map_err(|e| map_storage_error(&self.path, "write generation", e))?; + meta.remove(META_KEY_DUMP_TRANSACTION_HI) + .map_err(|e| map_storage_error(&self.path, "clear dump transaction", e))?; + meta.remove(META_KEY_DUMP_TRANSACTION_LO) + .map_err(|e| map_storage_error(&self.path, "clear dump transaction", e))?; } - tx.commit() - .map_err(|e| map_storage_error(&self.path, "commit transaction", e))?; + tx.commit().map_err(|e| map_commit_error(&self.path, e))?; Ok(()) } + /// Create a complete replacement database with an explicit committed generation. + /// + /// The caller must create this at a private staging path and publish it only after + /// independently verifying ownership of the live generation. + pub(crate) fn create_replacement( + path: PathBuf, + snapshot: &HashMap, + watermark: u64, + generation: u64, + ) -> Result<(), StorageError> { + Self::create_replacement_with_dump_transaction(path, snapshot, watermark, generation, None) + } + + pub(crate) fn create_dump_replacement( + path: PathBuf, + snapshot: &HashMap, + watermark: u64, + generation: u64, + transaction_id: &DumpTransactionId, + ) -> Result<(), StorageError> { + Self::create_replacement_with_dump_transaction( + path, + snapshot, + watermark, + generation, + Some(transaction_id), + ) + } + + fn create_replacement_with_dump_transaction( + path: PathBuf, + snapshot: &HashMap, + watermark: u64, + generation: u64, + dump_transaction_id: Option<&DumpTransactionId>, + ) -> Result<(), StorageError> { + let storage = Self::new(path); + storage.ensure_parent_dir()?; + let db = Database::create(&storage.path) + .map_err(|error| map_database_error(&storage.path, error))?; + let transaction = db + .begin_write() + .map_err(|error| map_tx_error(&storage.path, error, "begin replacement write"))?; + { + let mut metadata = transaction + .open_table(META) + .map_err(|error| map_table_error(&storage.path, error))?; + let mut mtimes = transaction + .open_table(FILES_MTIME) + .map_err(|error| map_table_error(&storage.path, error))?; + let mut hashes = transaction + .open_table(FILES_HASH) + .map_err(|error| map_table_error(&storage.path, error))?; + sync_file_tables(&storage.path, &mut mtimes, &mut hashes, snapshot)?; + metadata + .insert(META_KEY_WATERMARK, watermark) + .map_err(|error| map_storage_error(&storage.path, "write watermark", error))?; + metadata + .insert(META_KEY_GENERATION, generation) + .map_err(|error| map_storage_error(&storage.path, "write generation", error))?; + if let Some(transaction_id) = dump_transaction_id { + let value = transaction_id.as_u128(); + metadata + .insert(META_KEY_DUMP_TRANSACTION_HI, (value >> 64) as u64) + .map_err(|error| { + map_storage_error(&storage.path, "write dump transaction", error) + })?; + metadata + .insert(META_KEY_DUMP_TRANSACTION_LO, value as u64) + .map_err(|error| { + map_storage_error(&storage.path, "write dump transaction", error) + })?; + } + } + transaction + .commit() + .map_err(|error| map_commit_error(&storage.path, error)) + } + /// Read the current optimistic-lock generation. pub fn current_generation(&self) -> Result { - Ok(self.load_snapshot()?.generation) + Ok(match self.load_state()? { + HashStorageLoad::MissingPath | HashStorageLoad::ExistingUninitialized => 0, + HashStorageLoad::Initialized(snapshot) => snapshot.generation, + }) + } + + pub(crate) fn current_dump_transaction_id( + &self, + ) -> Result, StorageError> { + let database = match Database::open(&self.path) { + Ok(database) => database, + Err(database_error) => match self.path.try_exists() { + Ok(false) => return Ok(None), + Ok(true) => return Err(map_database_error(&self.path, database_error)), + Err(error) => return Err(map_filesystem_lookup_error(&self.path, error)), + }, + }; + let transaction = database + .begin_read() + .map_err(|error| map_tx_error(&self.path, error, "begin dump transaction read"))?; + let metadata = match transaction.open_table(META) { + Ok(metadata) => metadata, + Err(TableError::TableDoesNotExist(_)) => return Ok(None), + Err(error) => return Err(map_table_error(&self.path, error)), + }; + let high = metadata + .get(META_KEY_DUMP_TRANSACTION_HI) + .map_err(|error| map_storage_error(&self.path, "read dump transaction", error))? + .map(|value| value.value()); + let low = metadata + .get(META_KEY_DUMP_TRANSACTION_LO) + .map_err(|error| map_storage_error(&self.path, "read dump transaction", error))? + .map(|value| value.value()); + match (high, low) { + (None, None) => Ok(None), + (Some(high), Some(low)) => Ok(Some(DumpTransactionId::from_u128( + (u128::from(high) << 64) | u128::from(low), + ))), + _ => Err(StorageError::Recoverable { + path: self.path.clone(), + reason: "dump transaction metadata is incomplete".to_owned(), + }), + } } - /// Replace a corrupt or missing storage file with a fresh snapshot. - pub fn recover_and_commit_snapshot( + /// Read the optimistic-lock generation even when snapshot tables are recoverably incomplete. + pub fn recoverable_generation(&self) -> Result { + let database = + Database::open(&self.path).map_err(|error| map_database_error(&self.path, error))?; + let transaction = database + .begin_read() + .map_err(|error| map_tx_error(&self.path, error, "begin recovery read"))?; + let meta = match transaction.open_table(META) { + Ok(table) => Some(table), + Err(TableError::TableDoesNotExist(_)) => None, + Err(error) => return Err(map_table_error(&self.path, error)), + }; + read_generation(meta.as_ref(), &self.path) + } + + /// Commit against the exact storage state observed while preparing the snapshot. + pub fn commit_observed_snapshot( &self, snapshot: &HashMap, watermark: u64, + observed: &ObservedStorageState, ) -> Result<(), StorageError> { - if self.path.exists() { - let ts = std::time::SystemTime::now() - .duration_since(std::time::SystemTime::UNIX_EPOCH) - .map(|d| d.as_secs()) - .unwrap_or(0); - let backup = self.path.with_extension(format!("corrupt.{ts}.redb")); - std::fs::rename(&self.path, &backup).map_err(|e| StorageError::Hard { + match observed { + ObservedStorageState::MissingPath => { + if self + .path + .try_exists() + .map_err(|error| map_filesystem_lookup_error(&self.path, error))? + { + return Err(StorageError::ConcurrentStateModified { + path: self.path.clone(), + expected: 0, + actual: self.observed_generation_for_conflict()?, + }); + } + self.commit_missing_observation(snapshot, watermark) + } + ObservedStorageState::ExistingUninitialized { .. } => { + self.commit_claimed_observation(snapshot, watermark, observed, 0) + } + ObservedStorageState::Initialized { generation } => { + self.commit_snapshot(snapshot, watermark, *generation) + } + ObservedStorageState::Recoverable { generation, .. } => { + self.commit_claimed_observation(snapshot, watermark, observed, *generation) + } + } + } + + fn commit_missing_observation( + &self, + snapshot: &HashMap, + watermark: u64, + ) -> Result<(), StorageError> { + self.ensure_parent_dir()?; + let staged = self + .path + .with_extension(format!("missing-{}.redb", Uuid::new_v4())); + Self::create_replacement(staged.clone(), snapshot, watermark, 1)?; + #[cfg(test)] + BEFORE_MISSING_PUBLISH_HOOK.with(|cell| { + if let Some(hook) = cell.borrow_mut().take() { + hook(); + } + }); + match fs::hard_link(&staged, &self.path) { + Ok(()) => sync_parent_namespace(&self.path)?, + Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => { + cleanup_staged_file(&staged, "concurrent storage was preserved")?; + return Err(StorageError::ConcurrentStateModified { + path: self.path.clone(), + expected: 0, + actual: self.observed_generation_for_conflict()?, + }); + } + Err(error) => { + cleanup_staged_file(&staged, &format!("atomic publication failed ({error})"))?; + return Err(StorageError::Hard { + path: self.path.clone(), + reason: format!("failed to publish missing-state snapshot atomically: {error}"), + }); + } + } + cleanup_staged_file(&staged, "snapshot was published") + } + + fn commit_claimed_observation( + &self, + snapshot: &HashMap, + watermark: u64, + expected: &ObservedStorageState, + expected_generation: u64, + ) -> Result<(), StorageError> { + if matches!( + expected, + ObservedStorageState::MissingPath | ObservedStorageState::Initialized { .. } + ) { + return Err(StorageError::Hard { path: self.path.clone(), - reason: format!("failed to rename corrupt db to '{}': {e}", backup.display()), - })?; + reason: "claim requires an exact existing-file observation".to_owned(), + }); + } + self.ensure_parent_dir()?; + let suffix = Uuid::new_v4(); + let staged = self + .path + .with_extension(format!("replacement-{suffix}.redb")); + let claimed = self.path.with_extension(format!("corrupt-{suffix}.redb")); + Self::create_replacement( + staged.clone(), + snapshot, + watermark, + expected_generation + .checked_add(1) + .ok_or_else(|| StorageError::Hard { + path: self.path.clone(), + reason: "storage generation cannot advance beyond u64::MAX".to_owned(), + })?, + )?; + + #[cfg(test)] + BEFORE_RECOVERY_CLAIM_HOOK.with(|cell| { + if let Some(hook) = cell.borrow_mut().take() { + hook(); + } + }); + if let Err(claim_error) = fs::rename(&self.path, &claimed) { + let claim_error = StorageError::Hard { + path: self.path.clone(), + reason: format!("failed to claim observed recoverable storage: {claim_error}"), + }; + return match cleanup_staged_file(&staged, "storage claim failed") { + Ok(()) => Err(claim_error), + Err(cleanup_error) => Err(combine_storage_errors( + &self.path, + "storage claim and staging cleanup both failed", + claim_error, + cleanup_error, + )), + }; + } + if let Err(sync_error) = sync_parent_namespace(&self.path) { + let recovery = restore_and_cleanup(&claimed, &self.path, &staged); + return match recovery { + Ok(_) => Err(sync_error), + Err(recovery_error) => Err(combine_storage_errors( + &self.path, + "claim directory sync failed", + sync_error, + recovery_error, + )), + }; + } + let claimed_storage = Self::new(claimed.clone()); + let actual = match expected { + ObservedStorageState::ExistingUninitialized { .. } => { + claimed_storage.uninitialized_observation() + } + ObservedStorageState::Recoverable { .. } => claimed_storage.recoverable_observation(), + ObservedStorageState::MissingPath | ObservedStorageState::Initialized { .. } => { + restore_and_cleanup(&claimed, &self.path, &staged)?; + return Err(StorageError::Hard { + path: self.path.clone(), + reason: "invalid storage observation reached claim validation".to_owned(), + }); + } + }; + let actual_generation = actual.as_ref().ok().map(ObservedStorageState::generation); + if actual.as_ref().ok() != Some(expected) { + restore_and_cleanup(&claimed, &self.path, &staged)?; + return Err(StorageError::ConcurrentStateModified { + path: self.path.clone(), + expected: expected_generation, + actual: actual_generation, + }); + } + + #[cfg(test)] + AFTER_RECOVERY_CLAIM_HOOK.with(|cell| { + if let Some(hook) = cell.borrow_mut().take() { + hook(); + } + }); + let publication = publish_claimed_replacement(&staged, &self.path); + match publication { + Ok(()) => { + sync_parent_namespace(&self.path)?; + cleanup_staged_file(&staged, "replacement was published")?; + Ok(()) + } + Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => { + cleanup_staged_file(&staged, "concurrent storage was preserved")?; + Err(StorageError::ConcurrentStateModified { + path: self.path.clone(), + expected: expected_generation, + actual: self.observed_generation_for_conflict()?, + }) + } + Err(error) => { + #[cfg(test)] + BEFORE_RECOVERY_ROLLBACK_HOOK.with(|cell| { + if let Some(hook) = cell.borrow_mut().take() { + hook(); + } + }); + if restore_and_cleanup(&claimed, &self.path, &staged)? { + return Err(StorageError::ConcurrentStateModified { + path: self.path.clone(), + expected: expected_generation, + actual: self.observed_generation_for_conflict()?, + }); + } + Err(StorageError::Hard { + path: self.path.clone(), + reason: format!("failed to publish replacement: {error}"), + }) + } + } + } + + fn observed_generation_for_conflict(&self) -> Result, StorageError> { + match self.current_generation() { + Ok(generation) => Ok(Some(generation)), + Err(error) if error.is_recoverable() => Ok(None), + Err(error) => Err(error), } - self.commit_snapshot(snapshot, watermark, 0) } fn ensure_parent_dir(&self) -> Result<(), StorageError> { @@ -250,6 +813,168 @@ impl HashStorage { } } +fn storage_file_identity( + file: &fs::File, + metadata: &fs::Metadata, + path: &Path, +) -> Result { + #[cfg(unix)] + { + use std::os::unix::fs::MetadataExt; + let _ = file; + let _ = path; + return Ok(StorageFileIdentity { + len: metadata.len(), + device: metadata.dev(), + inode: metadata.ino(), + }); + } + #[cfg(windows)] + { + use std::mem::MaybeUninit; + use std::os::windows::io::AsRawHandle; + use windows_sys::Win32::Storage::FileSystem::{ + GetFileInformationByHandle, BY_HANDLE_FILE_INFORMATION, + }; + let mut information = MaybeUninit::::uninit(); + // SAFETY: the handle is borrowed from a live File; the API initializes + // the output structure on success. + let succeeded = unsafe { + GetFileInformationByHandle(file.as_raw_handle().cast(), information.as_mut_ptr()) + }; + if succeeded == 0 { + return Err(StorageError::Hard { + path: path.to_path_buf(), + reason: format!( + "query Windows storage file identity: {}", + std::io::Error::last_os_error() + ), + }); + } + // SAFETY: guarded by the successful Win32 call above. + let information = unsafe { information.assume_init() }; + return Ok(StorageFileIdentity { + len: metadata.len(), + volume_serial: information.dwVolumeSerialNumber, + file_index: (u64::from(information.nFileIndexHigh) << 32) + | u64::from(information.nFileIndexLow), + }); + } + #[cfg(not(any(unix, windows)))] + { + let _ = file; + let _ = path; + Ok(StorageFileIdentity { + len: metadata.len(), + }) + } +} + +#[cfg(windows)] +fn storage_metadata_is_reparse_point(metadata: &fs::Metadata) -> bool { + use std::os::windows::fs::MetadataExt; + const FILE_ATTRIBUTE_REPARSE_POINT: u32 = 0x0000_0400; + metadata.file_attributes() & FILE_ATTRIBUTE_REPARSE_POINT != 0 +} + +#[cfg(not(windows))] +fn storage_metadata_is_reparse_point(_metadata: &fs::Metadata) -> bool { + false +} + +#[cfg(test)] +fn run_test_hook( + hook: &'static std::thread::LocalKey>>>, +) { + hook.with(|cell| { + if let Some(hook) = cell.borrow_mut().take() { + hook(); + } + }); +} + +fn cleanup_staged_file(path: &Path, context: &str) -> Result<(), StorageError> { + fs::remove_file(path).map_err(|error| StorageError::Hard { + path: path.to_path_buf(), + reason: format!("{context}, but staging cleanup failed: {error}"), + })?; + sync_parent_namespace(path) +} + +/// Restore the claimed file without replacing a concurrently published target. +/// Returns `true` when a concurrent target was found and the claim was retained. +fn restore_claim_no_clobber(claimed: &Path, target: &Path) -> Result { + match fs::hard_link(claimed, target) { + Ok(()) => { + sync_parent_namespace(target)?; + fs::remove_file(claimed).map_err(|error| StorageError::Hard { + path: claimed.to_path_buf(), + reason: format!("claimed storage was restored but claim cleanup failed: {error}"), + })?; + sync_parent_namespace(claimed)?; + Ok(false) + } + Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => Ok(true), + Err(error) => Err(StorageError::Hard { + path: target.to_path_buf(), + reason: format!("failed to restore claimed storage without overwriting: {error}"), + }), + } +} + +fn restore_and_cleanup(claimed: &Path, target: &Path, staged: &Path) -> Result { + let restore = restore_claim_no_clobber(claimed, target); + let cleanup = cleanup_staged_file(staged, "claimed storage recovery was attempted"); + match (restore, cleanup) { + (Ok(concurrent), Ok(())) => Ok(concurrent), + (Err(error), Ok(())) | (Ok(_), Err(error)) => Err(error), + (Err(restore_error), Err(cleanup_error)) => Err(combine_storage_errors( + target, + "claim restoration and staging cleanup both failed", + restore_error, + cleanup_error, + )), + } +} + +fn combine_storage_errors( + path: &Path, + context: &str, + first: StorageError, + second: StorageError, +) -> StorageError { + StorageError::Hard { + path: path.to_path_buf(), + reason: format!("{context}: {first}; additionally: {second}"), + } +} + +fn publish_claimed_replacement(staged: &Path, target: &Path) -> std::io::Result<()> { + #[cfg(test)] + if FORCE_RECOVERY_PUBLISH_FAILURE.with(|forced| forced.replace(false)) { + return Err(std::io::Error::other("forced recovery publication failure")); + } + fs::hard_link(staged, target) +} + +#[cfg(unix)] +fn sync_parent_namespace(path: &Path) -> Result<(), StorageError> { + let Some(parent) = path.parent() else { + return Ok(()); + }; + fs::File::open(parent) + .and_then(|directory| directory.sync_all()) + .map_err(|error| StorageError::Hard { + path: parent.to_path_buf(), + reason: format!("failed to sync storage directory: {error}"), + }) +} + +#[cfg(not(unix))] +fn sync_parent_namespace(_path: &Path) -> Result<(), StorageError> { + Ok(()) +} + fn read_watermark( meta: Option<&redb::ReadOnlyTable<&str, u64>>, path: &Path, @@ -258,10 +983,7 @@ fn read_watermark( return Ok(None); }; meta.get(META_KEY_WATERMARK) - .map_err(|e| StorageError::Recoverable { - path: path.to_path_buf(), - reason: format!("read watermark: {e}"), - }) + .map_err(|e| map_storage_error(path, "read watermark", e)) .map(|opt| opt.map(|v| v.value())) } @@ -273,10 +995,7 @@ fn read_generation( return Ok(0); }; meta.get(META_KEY_GENERATION) - .map_err(|e| StorageError::Recoverable { - path: path.to_path_buf(), - reason: format!("read generation: {e}"), - }) + .map_err(|e| map_storage_error(path, "read generation", e)) .map(|opt| opt.map(|v| v.value()).unwrap_or(0)) } @@ -326,10 +1045,7 @@ fn map_database_error(path: &Path, err: DatabaseError) -> StorageError { path: path.to_path_buf(), reason: "previous I/O error in database".to_owned(), }, - DatabaseError::Storage(RedbStorageError::Io(e)) => StorageError::Hard { - path: path.to_path_buf(), - reason: format!("I/O error: {e}"), - }, + DatabaseError::Storage(RedbStorageError::Io(e)) => map_io_error(path, "I/O error", e), DatabaseError::DatabaseAlreadyOpen => StorageError::Hard { path: path.to_path_buf(), reason: "database is already open".to_owned(), @@ -343,10 +1059,7 @@ fn map_database_error(path: &Path, err: DatabaseError) -> StorageError { fn map_table_error(path: &Path, err: TableError) -> StorageError { match err { - TableError::Storage(RedbStorageError::Io(e)) => StorageError::Hard { - path: path.to_path_buf(), - reason: format!("table I/O error: {e}"), - }, + TableError::Storage(RedbStorageError::Io(e)) => map_io_error(path, "table I/O error", e), TableError::Storage(RedbStorageError::Corrupted(msg)) => StorageError::Recoverable { path: path.to_path_buf(), reason: msg, @@ -360,10 +1073,17 @@ fn map_table_error(path: &Path, err: TableError) -> StorageError { fn map_tx_error(path: &Path, err: TransactionError, context: &str) -> StorageError { match err { - TransactionError::Storage(RedbStorageError::Io(e)) => StorageError::Hard { + TransactionError::Storage(RedbStorageError::Io(e)) => map_io_error(path, context, e), + other => StorageError::Recoverable { path: path.to_path_buf(), - reason: format!("{context}: {e}"), + reason: format!("{context}: {other}"), }, + } +} + +fn map_storage_error(path: &Path, context: &str, err: RedbStorageError) -> StorageError { + match err { + RedbStorageError::Io(error) => map_io_error(path, context, error), other => StorageError::Recoverable { path: path.to_path_buf(), reason: format!("{context}: {other}"), @@ -371,9 +1091,333 @@ fn map_tx_error(path: &Path, err: TransactionError, context: &str) -> StorageErr } } -fn map_storage_error(path: &Path, context: &str, err: impl std::fmt::Display) -> StorageError { - StorageError::Recoverable { +fn map_io_error(path: &Path, context: &str, error: std::io::Error) -> StorageError { + let reason = format!("{context}: {error}"); + if error.kind() == std::io::ErrorKind::InvalidData { + StorageError::Recoverable { + path: path.to_path_buf(), + reason, + } + } else { + StorageError::Hard { + path: path.to_path_buf(), + reason, + } + } +} + +fn map_commit_error(path: &Path, error: CommitError) -> StorageError { + match error { + CommitError::Storage(error) => map_storage_error(path, "commit transaction", error), + other => StorageError::Recoverable { + path: path.to_path_buf(), + reason: format!("commit transaction: {other}"), + }, + } +} + +fn map_filesystem_lookup_error(path: &Path, error: std::io::Error) -> StorageError { + StorageError::Hard { path: path.to_path_buf(), - reason: format!("{context}: {err}"), + reason: format!("failed to inspect storage path: {error}"), + } +} + +#[cfg(test)] +mod tests { + use super::{ + HashStorage, HashStorageLoad, ObservedStorageState, StorageError, StoredFileState, + AFTER_RECOVERY_CLAIM_HOOK, BEFORE_EXACT_OBSERVATION_HOOK, BEFORE_MISSING_PUBLISH_HOOK, + BEFORE_RECOVERY_CLAIM_HOOK, BEFORE_RECOVERY_ROLLBACK_HOOK, FORCE_RECOVERY_PUBLISH_FAILURE, + META, META_KEY_GENERATION, + }; + use redb::Database; + use std::collections::HashMap; + use std::fs; + use tempfile::tempdir; + + #[test] + fn missing_file_has_typed_missing_state() { + let dir = tempdir().expect("tempdir"); + let storage = HashStorage::new(dir.path().join("missing.redb")); + + assert!(matches!( + storage.load_state(), + Ok(HashStorageLoad::MissingPath) + )); + } + + #[test] + fn existing_empty_database_has_typed_missing_state() { + let dir = tempdir().expect("tempdir"); + let path = dir.path().join("empty.redb"); + Database::create(&path).expect("empty database"); + + assert!(matches!( + HashStorage::new(path).load_state(), + Ok(HashStorageLoad::ExistingUninitialized) + )); + } + + #[test] + fn existing_empty_database_bootstraps_from_exact_observation() { + let dir = tempdir().expect("tempdir"); + let path = dir.path().join("empty.redb"); + let database = Database::create(&path).expect("empty database"); + drop(database); + let storage = HashStorage::new(path); + let observed = storage + .uninitialized_observation() + .expect("empty observation"); + + storage + .commit_observed_snapshot(&sample_snapshot("initialized"), 10, &observed) + .expect("bootstrap existing empty database"); + + let state = storage.load_state().expect("load initialized state"); + assert!(matches!( + state, + HashStorageLoad::Initialized(snapshot) if snapshot.generation == 1 + )); + } + + #[test] + fn empty_observation_rejects_concurrent_healthy_replacement() { + let dir = tempdir().expect("tempdir"); + let path = dir.path().join("state.redb"); + let database = Database::create(&path).expect("empty database"); + drop(database); + let storage = HashStorage::new(path.clone()); + assert!(matches!( + storage.load_state(), + Ok(HashStorageLoad::ExistingUninitialized) + )); + let healthy = dir.path().join("healthy.redb"); + HashStorage::create_replacement(healthy.clone(), &sample_snapshot("healthy"), 9, 9) + .expect("healthy database"); + let healthy_bytes = fs::read(&healthy).expect("healthy bytes"); + let target = path.clone(); + BEFORE_EXACT_OBSERVATION_HOOK.with(|cell| { + *cell.borrow_mut() = Some(Box::new(move || { + fs::remove_file(&target).expect("remove empty database"); + fs::rename(&healthy, &target).expect("publish healthy database"); + })); + }); + + let error = storage + .observe_state() + .expect_err("reclassification must reject concurrent repair"); + + assert!(matches!( + error, + StorageError::ConcurrentStateModified { .. } + )); + assert_eq!( + fs::read(path).expect("healthy database retained"), + healthy_bytes + ); + } + + #[test] + fn metadata_without_snapshot_tables_is_recoverable() { + let dir = tempdir().expect("tempdir"); + let path = dir.path().join("metadata-only.redb"); + let database = Database::create(&path).expect("database"); + let write = database.begin_write().expect("write"); + { + let mut meta = write.open_table(META).expect("meta"); + meta.insert(META_KEY_GENERATION, 1).expect("generation"); + } + write.commit().expect("commit"); + drop(database); + + assert!(matches!( + HashStorage::new(path).load_state(), + Err(StorageError::Recoverable { .. }) + )); + } + + #[cfg(unix)] + #[test] + fn filesystem_lookup_errors_are_hard_instead_of_missing() { + let dir = tempdir().expect("tempdir"); + let path = dir.path().join("loop.redb"); + std::os::unix::fs::symlink(&path, &path).expect("self symlink"); + + assert!(matches!( + HashStorage::new(path).load_state(), + Err(StorageError::Hard { .. }) + )); + } + + #[test] + fn dump_transaction_lookup_errors_are_hard_instead_of_missing() { + let dir = tempdir().expect("tempdir"); + let parent_file = dir.path().join("not-a-directory"); + fs::write(&parent_file, b"file").expect("parent file"); + let path = parent_file.join("state.redb"); + + assert!(matches!( + HashStorage::new(path).current_dump_transaction_id(), + Err(StorageError::Hard { .. }) + )); + } + + #[test] + fn exact_recovery_claim_preserves_concurrently_repaired_database() { + let dir = tempdir().expect("tempdir"); + let path = dir.path().join("state.redb"); + fs::write(&path, b"corrupt-a").expect("corrupt A"); + let storage = HashStorage::new(path.clone()); + let observed = storage.recoverable_observation().expect("observation A"); + let healthy = dir.path().join("healthy-b.redb"); + let snapshot = sample_snapshot("healthy"); + HashStorage::create_replacement(healthy.clone(), &snapshot, 9, 9).expect("healthy B"); + let healthy_bytes = fs::read(&healthy).expect("healthy bytes"); + let target = path.clone(); + BEFORE_RECOVERY_CLAIM_HOOK.with(|cell| { + *cell.borrow_mut() = Some(Box::new(move || { + fs::remove_file(&target).expect("remove A"); + fs::rename(&healthy, &target).expect("publish B"); + })); + }); + + let error = storage + .commit_observed_snapshot(&sample_snapshot("new"), 10, &observed) + .expect_err("concurrent repair must win"); + + assert!(matches!( + error, + StorageError::ConcurrentStateModified { .. } + )); + assert_eq!(fs::read(path).expect("preserved B"), healthy_bytes); + } + + #[test] + fn exact_recovery_claim_rejects_in_place_byte_change() { + let dir = tempdir().expect("tempdir"); + let path = dir.path().join("state.redb"); + fs::write(&path, b"corrupt-a").expect("corrupt A"); + let storage = HashStorage::new(path.clone()); + let observed = storage.recoverable_observation().expect("observation A"); + let target = path.clone(); + BEFORE_RECOVERY_CLAIM_HOOK.with(|cell| { + *cell.borrow_mut() = Some(Box::new(move || { + fs::write(&target, b"corrupt-b").expect("mutate A in place"); + })); + }); + + let error = storage + .commit_observed_snapshot(&sample_snapshot("new"), 10, &observed) + .expect_err("changed bytes must reject recovery"); + + assert!(matches!( + error, + StorageError::ConcurrentStateModified { .. } + )); + assert_eq!( + fs::read(path).expect("changed bytes retained"), + b"corrupt-b" + ); + } + + #[test] + fn missing_observation_does_not_overwrite_concurrent_initialization() { + let dir = tempdir().expect("tempdir"); + let path = dir.path().join("state.redb"); + let storage = HashStorage::new(path.clone()); + let healthy = dir.path().join("healthy-b.redb"); + HashStorage::create_replacement(healthy.clone(), &sample_snapshot("healthy"), 9, 9) + .expect("healthy B"); + let healthy_bytes = fs::read(&healthy).expect("healthy bytes"); + let target = path.clone(); + BEFORE_MISSING_PUBLISH_HOOK.with(|cell| { + *cell.borrow_mut() = Some(Box::new(move || { + fs::rename(&healthy, &target).expect("publish concurrent B"); + })); + }); + + let error = storage + .commit_observed_snapshot( + &sample_snapshot("new"), + 10, + &ObservedStorageState::MissingPath, + ) + .expect_err("concurrent initialization must win"); + + assert!(matches!( + error, + StorageError::ConcurrentStateModified { .. } + )); + assert_eq!(fs::read(path).expect("preserved B"), healthy_bytes); + } + + #[test] + fn recovery_publication_does_not_overwrite_database_published_after_claim() { + let dir = tempdir().expect("tempdir"); + let path = dir.path().join("state.redb"); + fs::write(&path, b"corrupt-a").expect("corrupt A"); + let storage = HashStorage::new(path.clone()); + let observed = storage.recoverable_observation().expect("observation A"); + let healthy = dir.path().join("healthy-b.redb"); + HashStorage::create_replacement(healthy.clone(), &sample_snapshot("healthy"), 9, 9) + .expect("healthy B"); + let healthy_bytes = fs::read(&healthy).expect("healthy bytes"); + let target = path.clone(); + AFTER_RECOVERY_CLAIM_HOOK.with(|cell| { + *cell.borrow_mut() = Some(Box::new(move || { + fs::hard_link(&healthy, &target).expect("publish B after claim"); + })); + }); + + let error = storage + .commit_observed_snapshot(&sample_snapshot("new"), 10, &observed) + .expect_err("concurrent publication must win"); + + assert!(matches!( + error, + StorageError::ConcurrentStateModified { .. } + )); + assert_eq!(fs::read(path).expect("preserved B"), healthy_bytes); + } + + #[test] + fn recovery_rollback_does_not_overwrite_database_published_before_restore() { + let dir = tempdir().expect("tempdir"); + let path = dir.path().join("state.redb"); + fs::write(&path, b"corrupt-a").expect("corrupt A"); + let storage = HashStorage::new(path.clone()); + let observed = storage.recoverable_observation().expect("observation A"); + let healthy = dir.path().join("healthy-b.redb"); + HashStorage::create_replacement(healthy.clone(), &sample_snapshot("healthy"), 9, 9) + .expect("healthy B"); + let healthy_bytes = fs::read(&healthy).expect("healthy bytes"); + let target = path.clone(); + FORCE_RECOVERY_PUBLISH_FAILURE.with(|forced| forced.set(true)); + BEFORE_RECOVERY_ROLLBACK_HOOK.with(|cell| { + *cell.borrow_mut() = Some(Box::new(move || { + fs::hard_link(&healthy, &target).expect("publish B before rollback"); + })); + }); + + let error = storage + .commit_observed_snapshot(&sample_snapshot("new"), 10, &observed) + .expect_err("rollback must preserve concurrent publication"); + + assert!(matches!( + error, + StorageError::ConcurrentStateModified { .. } + )); + assert_eq!(fs::read(path).expect("preserved B"), healthy_bytes); + } + + fn sample_snapshot(hash: &str) -> HashMap { + HashMap::from([( + "Configuration.xml".to_owned(), + StoredFileState { + mtime_ns: 1, + hash: hash.to_owned(), + }, + )]) } } diff --git a/src/change_detection/partial_load.rs b/src/change_detection/partial_load.rs index 61eb4bd..27ef23d 100644 --- a/src/change_detection/partial_load.rs +++ b/src/change_detection/partial_load.rs @@ -343,7 +343,10 @@ mod tests { let decision = decide( &[FileChange { path: module.clone(), + rel_path: "Catalogs.Items/ObjectModule.bsl".to_owned(), kind: ChangeKind::Modified, + pre_hash: Some("old".to_owned()), + post_hash: Some("new".to_owned()), }], root, DEFAULT_PARTIAL_LOAD_THRESHOLD, @@ -370,7 +373,10 @@ mod tests { let decision = decide( &[FileChange { path: module.clone(), + rel_path: "Catalogs/Items/Forms/ItemForm/Ext/Form/Module.bsl".to_owned(), kind: ChangeKind::Modified, + pre_hash: Some("old".to_owned()), + post_hash: Some("new".to_owned()), }], root, DEFAULT_PARTIAL_LOAD_THRESHOLD, @@ -394,7 +400,10 @@ mod tests { let decision = decide( &[FileChange { path: directory, + rel_path: "CommonModules".to_owned(), kind: ChangeKind::Modified, + pre_hash: Some("old".to_owned()), + post_hash: Some("new".to_owned()), }], root, DEFAULT_PARTIAL_LOAD_THRESHOLD, @@ -413,7 +422,10 @@ mod tests { let decision = decide( &[FileChange { path: config_xml, + rel_path: "Configuration.xml".to_owned(), kind: ChangeKind::Modified, + pre_hash: Some("old".to_owned()), + post_hash: Some("new".to_owned()), }], root, DEFAULT_PARTIAL_LOAD_THRESHOLD, @@ -431,7 +443,10 @@ mod tests { let decision = decide( &[FileChange { path: removed, + rel_path: "Catalogs.Items/ObjectModule.bsl".to_owned(), kind: ChangeKind::Deleted, + pre_hash: Some("old".to_owned()), + post_hash: None, }], root, DEFAULT_PARTIAL_LOAD_THRESHOLD, @@ -451,8 +466,11 @@ mod tests { std::fs::create_dir_all(path.parent().expect("parent")).expect("mkdir"); std::fs::write(&path, "module").expect("write"); changes.push(FileChange { + rel_path: format!("CommonModules/Module{index}.bsl"), path, kind: ChangeKind::Modified, + pre_hash: Some("old".to_owned()), + post_hash: Some("new".to_owned()), }); } @@ -479,7 +497,10 @@ mod tests { let decision = decide( &[FileChange { path: link_dir.join("ObjectModule.bsl"), + rel_path: "Catalogs.Items/ObjectModule.bsl".to_owned(), kind: ChangeKind::Modified, + pre_hash: Some("old".to_owned()), + post_hash: Some("new".to_owned()), }], &root, DEFAULT_PARTIAL_LOAD_THRESHOLD, diff --git a/src/change_detection/scanner.rs b/src/change_detection/scanner.rs index 04d83d9..ed5489f 100644 --- a/src/change_detection/scanner.rs +++ b/src/change_detection/scanner.rs @@ -1,6 +1,6 @@ use sha2::{Digest, Sha256}; use std::collections::HashSet; -use std::path::{Path, PathBuf}; +use std::path::{Component, Path, PathBuf}; use thiserror::Error; use walkdir::WalkDir; @@ -31,6 +31,93 @@ pub enum ScanError { #[error("failed to build path relative to scan root '{root}' for '{path}'")] RelativePath { root: PathBuf, path: PathBuf }, + + #[error("managed path relative to '{root}' is not valid UTF-8: '{path}'")] + NonUtf8RelativePath { root: PathBuf, path: PathBuf }, + + #[error( + "managed path relative to '{root}' contains a non-portable backslash component: '{path}'" + )] + NonPortableBackslash { root: PathBuf, path: PathBuf }, + + #[error("failed to inspect excluded source root '{path}': {source}")] + ExcludedRoot { + path: PathBuf, + source: std::io::Error, + }, + + #[error("excluded source root must be absolute: '{0}'")] + InvalidExcludedRoot(PathBuf), +} + +/// Precomputed policy defining which filesystem entries belong to a managed source tree. +#[derive(Debug, Clone)] +pub struct SourceInventoryPolicy { + active_excluded_roots: Vec, +} + +impl SourceInventoryPolicy { + pub fn new(root: &Path, excluded_roots: &[PathBuf]) -> Result { + let canonical_root = std::fs::canonicalize(root).map_err(|source| ScanError::Meta { + path: root.to_path_buf(), + source, + })?; + let mut active = Vec::new(); + for excluded in excluded_roots { + if !excluded.is_absolute() { + return Err(ScanError::InvalidExcludedRoot(excluded.clone())); + } + if excluded != root && excluded.starts_with(root) { + active.push(excluded.clone()); + } + match excluded.try_exists() { + Ok(true) => { + let canonical_excluded = std::fs::canonicalize(excluded).map_err(|source| { + ScanError::ExcludedRoot { + path: excluded.clone(), + source, + } + })?; + if canonical_excluded != canonical_root + && canonical_excluded.starts_with(&canonical_root) + { + if let Ok(suffix) = canonical_excluded.strip_prefix(&canonical_root) { + active.push(root.join(suffix)); + } + } + } + Ok(false) => {} + Err(source) => { + return Err(ScanError::ExcludedRoot { + path: excluded.clone(), + source, + }) + } + } + } + active.sort(); + active.dedup(); + Ok(Self { + active_excluded_roots: active, + }) + } + + pub fn excludes(&self, path: &Path) -> bool { + self.active_excluded_roots + .iter() + .any(|excluded| path.starts_with(excluded)) + } + + /// Whether a directory is part of the inventory traversal. + pub fn should_descend(&self, path: &Path) -> bool { + !self.excludes(path) && !has_ignored_name(path, IGNORED_DIRS) + } + + /// Whether a regular file belongs to the managed inventory. + /// This path-only predicate is reusable by staged/private inventory implementations. + pub fn includes_file(&self, path: &Path) -> bool { + !self.excludes(path) && !has_ignored_name(path, IGNORED_FILES) + } } /// Directory/file names that are always excluded from scanning. @@ -39,6 +126,32 @@ const IGNORED_DIRS: &[&str] = &[ ]; const IGNORED_FILES: &[&str] = &["ConfigDumpInfo.xml"]; +/// Whether a normalized relative path belongs to the repository-private inventory policy. +pub(crate) fn is_always_ignored_relative_path(path: &Path) -> bool { + let mut components = path.components().peekable(); + while let Some(component) = components.next() { + let Component::Normal(name) = component else { + return true; + }; + let text = name.to_string_lossy(); + if components.peek().is_some() + && IGNORED_DIRS + .iter() + .any(|ignored| text.eq_ignore_ascii_case(ignored)) + { + return true; + } + if components.peek().is_none() + && IGNORED_FILES + .iter() + .any(|ignored| text.eq_ignore_ascii_case(ignored)) + { + return true; + } + } + false +} + /// Coarse filesystem mtime guard (2 seconds). pub const COARSE_MARGIN_NS: u64 = 2_000_000_000; @@ -73,7 +186,9 @@ pub fn scan( root: &Path, watermark: Option, stored_keys: &HashSet, + excluded_roots: &[PathBuf], ) -> Result { + let policy = SourceInventoryPolicy::new(root, excluded_roots)?; let scan_started_at = mtime_nanos(std::time::SystemTime::now(), root).map_err(|source| ScanError::Mtime { path: root.to_path_buf(), @@ -86,7 +201,7 @@ pub fn scan( for entry in WalkDir::new(root) .follow_links(false) .into_iter() - .filter_entry(|e| !is_ignored_dir(e)) + .filter_entry(|entry| !entry.file_type().is_dir() || policy.should_descend(entry.path())) { let entry = entry.map_err(|e| ScanError::Walk { path: root.to_path_buf(), @@ -104,10 +219,8 @@ pub fn scan( } // Skip ignored file names. - if let Some(name) = path.file_name().and_then(|n| n.to_str()) { - if IGNORED_FILES.contains(&name) { - continue; - } + if !policy.includes_file(path) { + continue; } let meta = std::fs::metadata(path).map_err(|e| ScanError::Meta { @@ -145,6 +258,9 @@ pub fn scan( seen_files.push(seen); } + seen_files.sort_by(|left, right| left.rel_path.cmp(&right.rel_path)); + candidates.sort_by(|left, right| left.rel_path.cmp(&right.rel_path)); + Ok(ScanSnapshot { scan_started_at, seen_files, @@ -162,22 +278,131 @@ pub fn hash_file(path: &Path) -> Result { Ok(format!("{:x}", digest)) } -fn rel_path(root: &Path, path: &Path) -> Result { +/// Convert a path below `root` to the collision-free portable inventory representation. +pub fn portable_relative_path(root: &Path, path: &Path) -> Result { let rel = path .strip_prefix(root) .map_err(|_| ScanError::RelativePath { root: root.to_path_buf(), path: path.to_path_buf(), })?; - Ok(rel.to_string_lossy().replace('\\', "/")) + let mut normalized = Vec::new(); + for component in rel.components() { + let Component::Normal(component) = component else { + return Err(ScanError::RelativePath { + root: root.to_path_buf(), + path: path.to_path_buf(), + }); + }; + let component = component + .to_str() + .ok_or_else(|| ScanError::NonUtf8RelativePath { + root: root.to_path_buf(), + path: path.to_path_buf(), + })?; + if component.contains('\\') { + return Err(ScanError::NonPortableBackslash { + root: root.to_path_buf(), + path: path.to_path_buf(), + }); + } + normalized.push(component); + } + Ok(normalized.join("/")) +} + +fn rel_path(root: &Path, path: &Path) -> Result { + portable_relative_path(root, path) } -fn is_ignored_dir(entry: &walkdir::DirEntry) -> bool { - if !entry.file_type().is_dir() { - return false; +fn has_ignored_name(path: &Path, ignored_names: &[&str]) -> bool { + path.file_name() + .and_then(|name| name.to_str()) + .is_some_and(|name| { + ignored_names + .iter() + .any(|ignored| name.eq_ignore_ascii_case(ignored)) + }) +} + +#[cfg(test)] +mod tests { + use super::{rel_path, scan, ScanError}; + use std::collections::HashSet; + use tempfile::tempdir; + + #[test] + fn excludes_work_root_and_config_dump_info_case_insensitively() { + let dir = tempdir().expect("tempdir"); + let root = dir.path(); + let work = root.join("work"); + std::fs::create_dir(&work).expect("work"); + std::fs::write(root.join("Configuration.xml"), "managed").expect("managed"); + std::fs::write(root.join("configdumpinfo.XML"), "dump metadata").expect("dump info"); + std::fs::write(work.join("runtime.redb"), "state").expect("state"); + + let snapshot = scan(root, None, &HashSet::new(), &[work]).expect("scan"); + let paths = snapshot + .seen_files + .iter() + .map(|file| file.rel_path.as_str()) + .collect::>(); + + assert_eq!(paths, vec!["Configuration.xml"]); + } + + #[cfg(unix)] + #[test] + fn never_follows_symlinked_files_or_directories() { + let dir = tempdir().expect("tempdir"); + let root = dir.path().join("src"); + let outside = dir.path().join("outside"); + std::fs::create_dir(&root).expect("root"); + std::fs::create_dir(&outside).expect("outside"); + std::fs::write(outside.join("secret.xml"), "secret").expect("secret"); + std::os::unix::fs::symlink(&outside, root.join("linked-dir")).expect("dir link"); + std::os::unix::fs::symlink(outside.join("secret.xml"), root.join("linked-file.xml")) + .expect("file link"); + + let snapshot = scan(&root, None, &HashSet::new(), &[]).expect("scan"); + + assert!(snapshot.seen_files.is_empty()); + } + + #[test] + fn ancestor_exclusion_does_not_blank_generated_tree_inside_work_path() { + let dir = tempdir().expect("tempdir"); + let work = dir.path().join("work"); + let generated = work.join("designer/main"); + std::fs::create_dir_all(&generated).expect("generated"); + std::fs::write(generated.join("Configuration.xml"), "managed").expect("managed"); + + let snapshot = scan(&generated, None, &HashSet::new(), &[work]).expect("scan"); + + assert_eq!(snapshot.seen_files.len(), 1); + assert_eq!(snapshot.seen_files[0].rel_path, "Configuration.xml"); + } + + #[cfg(unix)] + #[test] + fn rejects_non_utf8_relative_paths_without_lossy_collisions() { + use std::ffi::OsString; + use std::os::unix::ffi::OsStringExt; + + let root = std::path::Path::new("/source"); + let path = root.join(OsString::from_vec(vec![b'a', 0xff])); + + let error = rel_path(root, &path).expect_err("non-utf8"); + assert!(matches!(error, ScanError::NonUtf8RelativePath { .. })); + } + + #[cfg(unix)] + #[test] + fn rejects_literal_backslash_in_unix_component() { + let dir = tempdir().expect("tempdir"); + std::fs::write(dir.path().join("a\\b.xml"), "managed").expect("file"); + + let error = scan(dir.path(), None, &HashSet::new(), &[]).expect_err("backslash"); + assert!(matches!(error, ScanError::NonPortableBackslash { .. })); } - let Some(name) = entry.file_name().to_str() else { - return false; - }; - IGNORED_DIRS.contains(&name) } diff --git a/src/change_detection/source_sets.rs b/src/change_detection/source_sets.rs index 6adb256..931bdc0 100644 --- a/src/change_detection/source_sets.rs +++ b/src/change_detection/source_sets.rs @@ -1,7 +1,10 @@ use std::path::{Path, PathBuf}; -use crate::change_detection::analyzer::{self, ContextAnalysis}; -use crate::config::model::{AppConfig, SourceFormat}; +use crate::config::model::{AppConfig, SourceFormat, SourceSetConfig}; +use crate::domain::runtime_state::{ + InfobaseIdentity, LogicalSourceRole, RuntimeSourceDescriptor, RuntimeSourceIdentityInputs, + RuntimeStateError, RuntimeStateLayout, +}; use crate::domain::source_set::SourceSetContext; /// Builds the list of [`SourceSetContext`] instances for the given config. @@ -11,11 +14,17 @@ use crate::domain::source_set::SourceSetContext; /// and a generated Designer copy under `workPath/designer//`. pub struct SourceSetsService<'a> { config: &'a AppConfig, + state_layout: RuntimeStateLayout, } impl<'a> SourceSetsService<'a> { - pub fn new(config: &'a AppConfig) -> Self { - Self { config } + pub fn new(config: &'a AppConfig) -> Result { + let identity = InfobaseIdentity::normalize(&config.infobase)?; + let state_layout = RuntimeStateLayout::new(&config.work_path, identity)?; + Ok(Self { + config, + state_layout, + }) } /// Return all Designer-format contexts that should be scanned and built. @@ -23,9 +32,9 @@ impl<'a> SourceSetsService<'a> { /// In `DESIGNER` mode this is simply each source-set resolved against the project base path. /// In `EDT` mode (Wave 2) this returns the generated Designer copies in /// `workPath/designer`. - pub fn designer_contexts(&self) -> Vec { - let base_path = absolutize_path(&self.config.base_path); - let work_path = absolutize_path(&self.config.work_path); + pub fn designer_contexts(&self) -> Result, RuntimeStateError> { + let base_path = absolutize_path(&self.config.base_path)?; + let work_path = absolutize_path(&self.config.work_path)?; match self.config.format { SourceFormat::Designer => self @@ -38,7 +47,7 @@ impl<'a> SourceSetsService<'a> { } else { base_path.join(&ss.path) }; - SourceSetContext::new(&ss.name, path, format!("designer-{}", ss.name)) + self.build_context(ss, path, LogicalSourceRole::DesignerSource, &work_path) }) .collect(), @@ -49,18 +58,19 @@ impl<'a> SourceSetsService<'a> { .map(|ss| { // Generated Designer copy lives at workPath/designer// let path = work_path.join("designer").join(&ss.name); - SourceSetContext::new(&ss.name, path, format!("designer-{}", ss.name)) + self.build_context(ss, path, LogicalSourceRole::DesignerSource, &work_path) }) .collect(), } } /// Return EDT source-set contexts (only meaningful in `EDT` format). - pub fn edt_contexts(&self) -> Vec { + pub fn edt_contexts(&self) -> Result, RuntimeStateError> { if self.config.format != SourceFormat::Edt { - return vec![]; + return Ok(vec![]); } - let base_path = absolutize_path(&self.config.base_path); + let base_path = absolutize_path(&self.config.base_path)?; + let work_path = absolutize_path(&self.config.work_path)?; self.config .source_sets .iter() @@ -70,24 +80,40 @@ impl<'a> SourceSetsService<'a> { } else { base_path.join(&ss.path) }; - SourceSetContext::new(&ss.name, path, format!("edt-{}", ss.name)) + self.build_context(ss, path, LogicalSourceRole::EdtSource, &work_path) }) .collect() } - /// Analyze all provided contexts and return context-tagged outcomes. - pub fn analyze_contexts(&self, contexts: &[SourceSetContext]) -> Vec { - analyzer::analyze_contexts(contexts, &self.config.work_path) + + fn build_context( + &self, + source_set: &SourceSetConfig, + source_root: PathBuf, + logical_role: LogicalSourceRole, + work_path: &Path, + ) -> Result { + let descriptor = RuntimeSourceDescriptor::new(RuntimeSourceIdentityInputs { + configured_source_identity: &source_set.path, + source_root: &source_root, + purpose: source_set.purpose, + format: self.config.format, + backend: self.config.builder, + logical_role, + })?; + let state = self + .state_layout + .source_state(&source_set.name, &descriptor); + Ok(SourceSetContext::new(&source_set.name, source_root, state) + .with_excluded_roots(vec![work_path.to_path_buf()])) } } -fn absolutize_path(path: &Path) -> PathBuf { +fn absolutize_path(path: &Path) -> Result { if path.is_absolute() { - return path.to_path_buf(); + return Ok(path.to_path_buf()); } - std::env::current_dir() - .expect("failed to resolve current working directory") - .join(path) + Ok(std::env::current_dir()?.join(path)) } #[cfg(test)] @@ -119,8 +145,8 @@ mod tests { tests: TestsConfig::default(), }; - let service = SourceSetsService::new(&config); - let contexts = service.designer_contexts(); + let service = SourceSetsService::new(&config).expect("service"); + let contexts = service.designer_contexts().expect("contexts"); assert_eq!(contexts.len(), 1); assert!(contexts[0].path().is_absolute()); @@ -147,8 +173,8 @@ mod tests { tests: TestsConfig::default(), }; - let service = SourceSetsService::new(&config); - let contexts = service.designer_contexts(); + let service = SourceSetsService::new(&config).expect("service"); + let contexts = service.designer_contexts().expect("contexts"); assert_eq!(contexts.len(), 1); assert!(contexts[0] diff --git a/src/domain/build.rs b/src/domain/build.rs index 06abd99..0daae4c 100644 --- a/src/domain/build.rs +++ b/src/domain/build.rs @@ -1,3 +1,4 @@ +use crate::domain::sync_receipt::SyncReceipt; use serde::{Deserialize, Serialize}; #[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] @@ -14,6 +15,8 @@ pub struct BuildStep { pub ok: bool, pub message: Option, pub duration_ms: u64, + #[serde(default)] + pub receipt: SyncReceipt, } #[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] diff --git a/src/domain/dump.rs b/src/domain/dump.rs index 34117e6..cb0ca5e 100644 --- a/src/domain/dump.rs +++ b/src/domain/dump.rs @@ -1,3 +1,4 @@ +use crate::domain::sync_receipt::SyncReceipt; use serde::{Deserialize, Serialize}; use std::path::PathBuf; @@ -16,6 +17,8 @@ pub struct DumpResult { pub duration_ms: u64, #[serde(skip_serializing_if = "Option::is_none")] pub message: Option, + #[serde(default)] + pub receipt: SyncReceipt, } #[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] diff --git a/src/domain/mod.rs b/src/domain/mod.rs index 054ae3d..56903db 100644 --- a/src/domain/mod.rs +++ b/src/domain/mod.rs @@ -28,8 +28,12 @@ pub mod load; pub mod partial_dump_selector; /// Shared runner models. pub mod runner; +/// Versioned runtime-state identity and path models. +pub mod runtime_state; /// Source set domain models. pub mod source_set; +/// Typed terminal synchronization receipts shared by build and dump workflows. +pub mod sync_receipt; /// Syntax-check domain models. pub mod syntax; /// Test domain models. diff --git a/src/domain/runtime_state.rs b/src/domain/runtime_state.rs new file mode 100644 index 0000000..4572808 --- /dev/null +++ b/src/domain/runtime_state.rs @@ -0,0 +1,1210 @@ +use std::fmt; +use std::path::{Path, PathBuf}; + +use serde::{Deserialize, Deserializer, Serialize, Serializer}; +use sha2::{Digest, Sha256}; +use thiserror::Error; + +use crate::config::model::{BuilderBackend, InfobaseConfig, SourceFormat, SourceSetPurpose}; +use crate::support::connection_args::split_v8_arg_string; +use crate::support::path::nearest_existing_canonical_path; + +const STATE_NAMESPACE: &str = "v8-runner/runtime-state/v1"; + +/// Durable identity tying one source publication to its exact runtime-state commit. +#[derive(Debug, Clone, PartialEq, Eq, Hash)] +pub(crate) struct DumpTransactionId(String); + +impl DumpTransactionId { + pub(crate) fn new() -> Self { + Self(uuid::Uuid::new_v4().hyphenated().to_string()) + } + + pub(crate) fn from_u128(value: u128) -> Self { + Self(uuid::Uuid::from_u128(value).hyphenated().to_string()) + } + + pub(crate) fn as_u128(&self) -> u128 { + uuid::Uuid::parse_str(&self.0) + .expect("DumpTransactionId invariant") + .as_u128() + } +} + +impl fmt::Display for DumpTransactionId { + fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { + formatter.write_str(&self.0) + } +} + +impl Serialize for DumpTransactionId { + fn serialize(&self, serializer: S) -> Result + where + S: Serializer, + { + serializer.serialize_str(&self.0) + } +} + +impl<'de> Deserialize<'de> for DumpTransactionId { + fn deserialize(deserializer: D) -> Result + where + D: Deserializer<'de>, + { + let value = String::deserialize(deserializer)?; + let uuid = uuid::Uuid::parse_str(&value).map_err(serde::de::Error::custom)?; + let canonical = uuid.hyphenated().to_string(); + if canonical != value { + return Err(serde::de::Error::custom( + "dump transaction id must be a canonical UUID", + )); + } + Ok(Self(canonical)) + } +} + +/// A normalized, secret-free identity of one target infobase. +#[derive(Clone, PartialEq, Eq)] +pub struct InfobaseIdentity { + connection_fingerprint: String, + fingerprint: String, + kind: InfobaseKind, +} + +impl InfobaseIdentity { + /// Normalize all supported 1C connection forms and exclude authentication material. + pub fn normalize(config: &InfobaseConfig) -> Result { + let connection = NormalizedConnection::parse(&config.connection)?; + let connection_fingerprint = tagged_hash(&connection.fields()); + let mut identity_fields = vec![ + ("namespace", STATE_NAMESPACE.as_bytes().to_vec()), + ("connection", connection_fingerprint.as_bytes().to_vec()), + ]; + if let Some(dbms) = &config.dbms { + for (tag, value) in [ + ("dbms-kind", dbms.kind.as_deref()), + ("dbms-server", dbms.server.as_deref()), + ("dbms-name", dbms.name.as_deref()), + ] { + if let Some(value) = value.map(str::trim).filter(|value| !value.is_empty()) { + identity_fields.push((tag, value.to_ascii_lowercase().into_bytes())); + } + } + } + + Ok(Self { + connection_fingerprint, + fingerprint: tagged_hash(&identity_fields), + kind: connection.kind(), + }) + } + + pub fn fingerprint(&self) -> &str { + &self.fingerprint + } + + #[cfg(test)] + pub fn connection_fingerprint(&self) -> &str { + &self.connection_fingerprint + } +} + +impl fmt::Debug for InfobaseIdentity { + fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { + formatter + .debug_struct("InfobaseIdentity") + .field("kind", &self.kind) + .field("fingerprint", &self.fingerprint) + .finish() + } +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum InfobaseKind { + File, + Server, +} + +enum NormalizedConnection { + File(PathBuf), + Server(Vec<(String, String)>), +} + +impl NormalizedConnection { + fn parse(raw: &str) -> Result { + let trimmed = raw.trim(); + if trimmed.is_empty() { + return Err(RuntimeStateError::EmptyConnection); + } + if trimmed.starts_with('/') || trimmed.starts_with('-') { + return Self::parse_raw(trimmed); + } + Self::parse_connection_string(trimmed) + } + + fn parse_raw(raw: &str) -> Result { + let tokens = split_raw_args(raw)?; + let mut connection: Option = None; + let mut index = 0; + while index < tokens.len() { + let flag = tokens[index].as_str(); + if is_auth_flag(flag) { + index += 1; + if index >= tokens.len() { + return Err(RuntimeStateError::MalformedRawConnection); + } + index += 1; + continue; + } + if connection.is_some() || index + 1 >= tokens.len() { + return Err(RuntimeStateError::UnsupportedRawConnection); + } + let value = &tokens[index + 1]; + connection = Some( + if flag.eq_ignore_ascii_case("/f") || flag.eq_ignore_ascii_case("-f") { + Self::File(canonicalize_stable(Path::new(value))?) + } else if flag.eq_ignore_ascii_case("/s") || flag.eq_ignore_ascii_case("-s") { + let (server, reference) = value + .split_once('\\') + .ok_or(RuntimeStateError::MalformedRawConnection)?; + if server.trim().is_empty() || reference.trim().is_empty() { + return Err(RuntimeStateError::MalformedRawConnection); + } + Self::Server(vec![ + ("ref".to_owned(), reference.trim().to_ascii_lowercase()), + ("srvr".to_owned(), server.trim().to_ascii_lowercase()), + ]) + } else if flag.eq_ignore_ascii_case("/ibconnectionstring") + || flag.eq_ignore_ascii_case("-ibconnectionstring") + { + Self::parse_connection_string(value)? + } else { + return Err(RuntimeStateError::UnsupportedRawConnection); + }, + ); + index += 2; + } + connection.ok_or(RuntimeStateError::MalformedRawConnection) + } + + fn parse_connection_string(raw: &str) -> Result { + let mut fields = Vec::new(); + let mut file_path = None; + for item in split_connection_fields(raw)? { + let (key, value) = item + .split_once('=') + .ok_or(RuntimeStateError::MalformedConnectionString)?; + let key = key.trim().to_ascii_lowercase(); + if is_credential_key(&key) { + continue; + } + let value = value.trim(); + if key == "file" { + file_path = Some(canonicalize_stable(Path::new(value))?); + continue; + } + fields.push((key, value.to_ascii_lowercase())); + } + if let Some(file_path) = file_path { + return Ok(Self::File(file_path)); + } + if fields.is_empty() { + return Err(RuntimeStateError::MalformedConnectionString); + } + fields.sort(); + Ok(Self::Server(fields)) + } + + fn kind(&self) -> InfobaseKind { + match self { + Self::File(_) => InfobaseKind::File, + Self::Server(_) => InfobaseKind::Server, + } + } + + fn fields(&self) -> Vec<(&'static str, Vec)> { + match self { + Self::File(path) => vec![("kind", b"file".to_vec()), ("path", path_bytes(path))], + Self::Server(fields) => { + let mut result = vec![("kind", b"server".to_vec())]; + for (key, value) in fields { + result.push(("field-key", key.as_bytes().to_vec())); + result.push(("field-value", value.as_bytes().to_vec())); + } + result + } + } + } +} + +/// Logical role separates configured, generated and tool-owned views of one source-set. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum LogicalSourceRole { + DesignerSource, + EdtSource, + ToolExtension, +} + +impl LogicalSourceRole { + fn label(self) -> &'static str { + match self { + Self::DesignerSource => "designer-source", + Self::EdtSource => "edt-source", + Self::ToolExtension => "tool-extension", + } + } +} + +/// Identity inputs for one logical source view. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct RuntimeSourceDescriptor { + fingerprint: String, +} + +/// Named inputs prevent format/backend/purpose fields from being accidentally swapped. +#[derive(Debug, Clone, Copy)] +pub struct RuntimeSourceIdentityInputs<'a> { + pub configured_source_identity: &'a Path, + pub source_root: &'a Path, + pub purpose: SourceSetPurpose, + pub format: SourceFormat, + pub backend: BuilderBackend, + pub logical_role: LogicalSourceRole, +} + +impl RuntimeSourceDescriptor { + pub fn new(inputs: RuntimeSourceIdentityInputs<'_>) -> Result { + let canonical_root = canonicalize_stable(inputs.source_root)?; + let fields = vec![ + ( + "configured-source", + path_bytes(inputs.configured_source_identity), + ), + ("canonical-root", path_bytes(&canonical_root)), + ("purpose", purpose_label(inputs.purpose).as_bytes().to_vec()), + ("format", format_label(inputs.format).as_bytes().to_vec()), + ("backend", backend_label(inputs.backend).as_bytes().to_vec()), + ( + "logical-role", + inputs.logical_role.label().as_bytes().to_vec(), + ), + ]; + Ok(Self { + fingerprint: tagged_hash(&fields), + }) + } + + pub fn fingerprint(&self) -> &str { + &self.fingerprint + } +} + +/// Versioned root for all state associated with one infobase. +#[derive(Debug, Clone)] +pub struct RuntimeStateLayout { + ib_state_dir: PathBuf, +} + +impl RuntimeStateLayout { + pub fn new( + work_path: impl AsRef, + identity: InfobaseIdentity, + ) -> Result { + let work_path = canonicalize_stable(work_path.as_ref())?; + let infobase_fingerprint = identity.fingerprint().to_owned(); + Ok(Self { + ib_state_dir: work_path + .join("ib-state") + .join("v1") + .join(&infobase_fingerprint), + }) + } + + pub fn source_state( + &self, + source_set: &str, + descriptor: &RuntimeSourceDescriptor, + ) -> RuntimeSourceState { + let safe_name = sanitize_source_set(source_set); + let context_fingerprint = tagged_hash(&[ + ("source-set-name", source_set.as_bytes().to_vec()), + ( + "source-identity", + descriptor.fingerprint().as_bytes().to_vec(), + ), + ]); + RuntimeSourceState { + state_dir: self + .ib_state_dir + .join(format!("{safe_name}-{context_fingerprint}")), + context_fingerprint, + } + } +} + +/// Paths owned by one source view inside one infobase namespace. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct RuntimeSourceState { + state_dir: PathBuf, + context_fingerprint: String, +} + +impl RuntimeSourceState { + #[allow(dead_code)] // Foundation consumed by the private CDFI/baseline tasks in this plan. + pub fn state_dir(&self) -> &Path { + &self.state_dir + } + + pub fn hash_storage_path(&self) -> PathBuf { + self.state_dir.join("hash-storage.redb") + } + + #[cfg(test)] + pub fn context_fingerprint(&self) -> &str { + &self.context_fingerprint + } + + #[allow(dead_code)] // Foundation consumed by the private Designer transaction task. + pub fn private_cdfi_path(&self) -> PathBuf { + self.state_dir.join("ConfigDumpInfo.xml") + } + + #[allow(dead_code)] // Foundation consumed by the recoverable publication task. + pub fn transactions_dir(&self) -> PathBuf { + self.state_dir.join("transactions") + } + + #[allow(dead_code)] // Typed handle intentionally lands before its persistence implementation. + pub fn baseline(&self, role: BaselineRole, generation: StateGeneration) -> IbBaseline { + IbBaseline { + path: self + .state_dir + .join("generations") + .join(generation.value.to_string()) + .join("ib-baseline") + .join(role.label()), + generation, + role, + } + } + + #[allow(dead_code)] + pub fn ib_baseline(&self, generation: StateGeneration) -> IbBaseline { + self.baseline(BaselineRole::ConfiguredSource, generation) + } + + #[allow(dead_code)] // Typed handle intentionally lands before its persistence implementation. + pub fn source_observation(&self, generation: StateGeneration) -> SourceObservation { + SourceObservation { + path: self + .state_dir + .join("generations") + .join(generation.value.to_string()) + .join("source-observation"), + generation, + } + } +} + +/// Optimistic generation shared by the typed runtime-state handles. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +#[allow(dead_code)] // Foundation consumed by later runtime-state generation commits. +pub struct StateGeneration { + value: u64, +} + +/// Semantic origin of a complete private baseline. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum BaselineRole { + /// The platform dump corresponding to the configured source tree. + ConfiguredSource, + /// The intermediate Designer tree produced for an EDT source set. + EdtPlatformDesigner, +} + +impl BaselineRole { + const fn label(self) -> &'static str { + match self { + Self::ConfiguredSource => "configured-source", + Self::EdtPlatformDesigner => "edt-platform-designer", + } + } +} + +impl StateGeneration { + #[allow(dead_code)] + pub const fn new(value: u64) -> Self { + Self { value } + } + + #[allow(dead_code)] + pub const fn value(self) -> u64 { + self.value + } +} + +/// Complete private infobase baseline for one state generation. +#[derive(Debug, Clone, PartialEq, Eq)] +#[allow(dead_code)] // Distinct type prevents baseline/observation path mix-ups in later tasks. +pub struct IbBaseline { + path: PathBuf, + generation: StateGeneration, + role: BaselineRole, +} + +impl IbBaseline { + #[allow(dead_code)] + pub fn path(&self) -> &Path { + &self.path + } + + #[allow(dead_code)] + pub const fn generation(&self) -> StateGeneration { + self.generation + } + + #[cfg(test)] + pub const fn role(&self) -> BaselineRole { + self.role + } +} + +/// Last successfully observed source snapshot for one state generation. +#[derive(Debug, Clone, PartialEq, Eq)] +#[allow(dead_code)] // Distinct type prevents baseline/observation path mix-ups in later tasks. +pub struct SourceObservation { + path: PathBuf, + generation: StateGeneration, +} + +impl SourceObservation { + #[allow(dead_code)] + pub fn path(&self) -> &Path { + &self.path + } + + #[allow(dead_code)] + pub const fn generation(&self) -> StateGeneration { + self.generation + } +} + +#[derive(Debug, Error)] +pub enum RuntimeStateError { + #[error("infobase connection is empty")] + EmptyConnection, + #[error("infobase connection string is malformed")] + MalformedConnectionString, + #[error("raw infobase connection is malformed")] + MalformedRawConnection, + #[error("raw infobase connection form is unsupported")] + UnsupportedRawConnection, + #[error("runtime identity path cannot be resolved: {0}")] + PathResolution(#[from] std::io::Error), +} + +fn split_raw_args(raw: &str) -> Result, RuntimeStateError> { + let (tokens, balanced_quotes) = split_v8_arg_string(raw); + if !balanced_quotes { + return Err(RuntimeStateError::MalformedRawConnection); + } + Ok(tokens) +} + +fn is_auth_flag(value: &str) -> bool { + ["/n", "-n", "/p", "-p"] + .iter() + .any(|flag| value.eq_ignore_ascii_case(flag)) +} + +fn is_credential_key(value: &str) -> bool { + matches!( + value, + "usr" + | "user" + | "username" + | "pwd" + | "password" + | "dbuser" + | "dbusr" + | "dbpassword" + | "dbpwd" + ) +} + +fn split_connection_fields(raw: &str) -> Result, RuntimeStateError> { + let mut fields = Vec::new(); + let mut current = String::new(); + let mut quoted = false; + let mut characters = raw.chars().peekable(); + while let Some(character) = characters.next() { + match character { + '"' if quoted && characters.peek() == Some(&'"') => { + characters.next(); + current.push('"'); + } + '"' => quoted = !quoted, + ';' if !quoted => { + let field = current.trim(); + if !field.is_empty() { + fields.push(field.to_owned()); + } + current.clear(); + } + value => current.push(value), + } + } + if quoted { + return Err(RuntimeStateError::MalformedConnectionString); + } + let field = current.trim(); + if !field.is_empty() { + fields.push(field.to_owned()); + } + Ok(fields) +} + +fn canonicalize_stable(path: &Path) -> Result { + nearest_existing_canonical_path(path).map_err(RuntimeStateError::from) +} + +#[cfg(unix)] +fn path_bytes(path: &Path) -> Vec { + use std::os::unix::ffi::OsStrExt; + path.as_os_str().as_bytes().to_vec() +} + +/// Windows paths are hashed as deterministic little-endian UTF-16 code units. +/// Case is preserved: the runtime layout does not guess filesystem case semantics. +#[cfg(windows)] +fn path_bytes(path: &Path) -> Vec { + use std::os::windows::ffi::OsStrExt; + path.as_os_str() + .encode_wide() + .flat_map(u16::to_le_bytes) + .collect() +} + +fn tagged_hash(fields: &[(&str, Vec)]) -> String { + let mut hasher = Sha256::new(); + for (tag, value) in fields { + hasher.update((tag.len() as u64).to_be_bytes()); + hasher.update(tag.as_bytes()); + hasher.update((value.len() as u64).to_be_bytes()); + hasher.update(value); + } + format!("{:x}", hasher.finalize()) +} + +fn purpose_label(value: SourceSetPurpose) -> &'static str { + match value { + SourceSetPurpose::Configuration => "configuration", + SourceSetPurpose::Extension => "extension", + SourceSetPurpose::ExternalDataProcessors => "external-data-processors", + SourceSetPurpose::ExternalReports => "external-reports", + } +} + +fn format_label(value: SourceFormat) -> &'static str { + match value { + SourceFormat::Designer => "designer", + SourceFormat::Edt => "edt", + } +} + +fn backend_label(value: BuilderBackend) -> &'static str { + match value { + BuilderBackend::Designer => "designer", + BuilderBackend::Ibcmd => "ibcmd", + } +} + +fn sanitize_source_set(value: &str) -> String { + let sanitized: String = value + .chars() + .map(|character| { + if character.is_ascii_alphanumeric() || matches!(character, '-' | '_') { + character + } else { + '_' + } + }) + .collect(); + let sanitized = sanitized.trim_matches(['.', '_', '-']); + if sanitized.is_empty() { + "source-set".to_owned() + } else { + sanitized.chars().take(64).collect() + } +} + +#[cfg(test)] +mod tests { + use super::{ + BaselineRole, InfobaseIdentity, LogicalSourceRole, RuntimeSourceDescriptor, + RuntimeSourceIdentityInputs, RuntimeStateLayout, StateGeneration, + }; + use crate::config::model::{ + BuilderBackend, InfobaseConfig, InfobaseDbmsConfig, SourceFormat, SourceSetPurpose, + }; + use std::path::{Path, PathBuf}; + use tempfile::tempdir; + + fn file_infobase(connection: impl Into) -> InfobaseConfig { + InfobaseConfig { + connection: connection.into(), + user: None, + password: None, + dbms: None, + } + } + + #[test] + fn equivalent_plain_and_raw_file_connections_have_one_identity() { + let dir = tempdir().expect("tempdir"); + let ib = dir.path().join("ib"); + std::fs::create_dir(&ib).expect("ib"); + let plain = file_infobase(format!("File={}", ib.display())); + let raw_slash = file_infobase(format!("/F \"{}\"", ib.display())); + let raw_dash = file_infobase(format!("-F {}", ib.display())); + let wrapped = file_infobase(format!("/IBConnectionString File={}", ib.display())); + + let expected = InfobaseIdentity::normalize(&plain).expect("plain identity"); + assert_eq!( + expected.fingerprint(), + InfobaseIdentity::normalize(&raw_slash) + .expect("slash identity") + .fingerprint() + ); + assert_eq!( + expected.fingerprint(), + InfobaseIdentity::normalize(&raw_dash) + .expect("dash identity") + .fingerprint() + ); + assert_eq!( + expected.fingerprint(), + InfobaseIdentity::normalize(&wrapped) + .expect("wrapped identity") + .fingerprint() + ); + } + + #[test] + fn quoted_semicolon_file_path_is_not_split_as_a_field_separator() { + let dir = tempdir().expect("tempdir"); + let semicolon_path = dir.path().join("ib;accounting"); + let plain = file_infobase(format!("File=\"{}\"", semicolon_path.display())); + let raw = file_infobase(format!("/F \"{}\"", semicolon_path.display())); + let different = file_infobase(format!("File={}", dir.path().join("ib").display())); + + let plain = InfobaseIdentity::normalize(&plain).expect("quoted identity"); + assert_eq!( + plain.fingerprint(), + InfobaseIdentity::normalize(&raw) + .expect("raw identity") + .fingerprint() + ); + assert_ne!( + plain.fingerprint(), + InfobaseIdentity::normalize(&different) + .expect("different identity") + .fingerprint() + ); + } + + #[test] + fn doubled_quotes_are_accepted_inside_plain_connection_values() { + InfobaseIdentity::normalize(&file_infobase("File=\"ib\"\"quoted\"")) + .expect("doubled quote"); + } + + #[test] + fn raw_windows_path_with_trailing_separator_uses_execution_quote_semantics() { + let raw = file_infobase(r#"/F "C:\work\ib\""#); + let plain = file_infobase(r#"File="C:\work\ib\""#); + + assert_eq!( + InfobaseIdentity::normalize(&raw) + .expect("raw identity") + .fingerprint(), + InfobaseIdentity::normalize(&plain) + .expect("plain identity") + .fingerprint() + ); + } + + #[cfg(windows)] + #[test] + fn windows_nonexistent_path_suffix_preserves_case_in_identity() { + let dir = tempdir().expect("tempdir"); + let upper = file_infobase(format!("File={}\\FutureIB", dir.path().display())); + let lower = file_infobase(format!("File={}\\futureib", dir.path().display())); + + assert_ne!( + InfobaseIdentity::normalize(&upper) + .expect("upper") + .fingerprint(), + InfobaseIdentity::normalize(&lower) + .expect("lower") + .fingerprint() + ); + } + + #[test] + fn malformed_quotes_and_trailing_fields_are_rejected() { + for connection in [ + "File=\"/tmp/unterminated", + r#"File="ib\"quoted""#, + "File=/tmp/ib;broken", + "/F /tmp/ib /N", + "/F /tmp/ib /P", + ] { + assert!( + InfobaseIdentity::normalize(&file_infobase(connection)).is_err(), + "accepted malformed connection: {connection}" + ); + } + } + + #[test] + fn equivalent_plain_and_raw_server_connections_have_one_identity() { + let plain = file_infobase("Srvr=Demo;Ref=Accounting"); + let raw = file_infobase("/S Demo\\Accounting"); + + assert_eq!( + InfobaseIdentity::normalize(&plain) + .expect("plain identity") + .fingerprint(), + InfobaseIdentity::normalize(&raw) + .expect("raw identity") + .fingerprint() + ); + } + + #[test] + fn nearest_existing_parent_makes_nonexistent_paths_stable() { + let dir = tempdir().expect("tempdir"); + let existing = dir.path().join("existing"); + std::fs::create_dir(&existing).expect("existing"); + let through_dot_segments = existing.join("missing").join("..").join("future-ib"); + let direct = existing.join("future-ib"); + + assert_eq!( + InfobaseIdentity::normalize(&file_infobase(format!( + "File={}", + through_dot_segments.display() + ))) + .expect("first identity") + .fingerprint(), + InfobaseIdentity::normalize(&file_infobase(format!("File={}", direct.display()))) + .expect("second identity") + .fingerprint() + ); + } + + #[cfg(unix)] + #[test] + fn symlinked_file_paths_resolve_to_the_same_identity() { + use std::os::unix::fs::symlink; + + let dir = tempdir().expect("tempdir"); + let real = dir.path().join("real"); + let link = dir.path().join("link"); + std::fs::create_dir(&real).expect("real"); + symlink(&real, &link).expect("symlink"); + + assert_eq!( + InfobaseIdentity::normalize(&file_infobase(format!("File={}", real.display()))) + .expect("real identity") + .fingerprint(), + InfobaseIdentity::normalize(&file_infobase(format!("File={}", link.display()))) + .expect("link identity") + .fingerprint() + ); + } + + #[test] + fn credentials_do_not_affect_identity_or_debug_output() { + let clean = file_infobase("Srvr=demo;Ref=accounting"); + let with_secrets = InfobaseConfig { + connection: "Srvr=demo;Ref=accounting;Usr=alice;Pwd=hunter2;DBUser=sa;DBPwd=db-secret" + .to_owned(), + user: Some("top-secret-user".to_owned()), + password: Some("top-secret-password".to_owned()), + dbms: Some( + InfobaseDbmsConfig::new("PostgreSQL", "db", "accounting") + .with_credentials(Some("db-admin".to_owned()), Some("db-password".to_owned())), + ), + }; + let clean = InfobaseIdentity::normalize(&clean).expect("clean identity"); + let secret = InfobaseIdentity::normalize(&with_secrets).expect("secret identity"); + let debug = format!("{secret:?}"); + + assert_eq!( + clean.connection_fingerprint(), + secret.connection_fingerprint() + ); + for value in [ + "alice", + "hunter2", + "sa", + "db-secret", + "top-secret-user", + "top-secret-password", + "db-admin", + "db-password", + "Srvr=demo", + ] { + assert!(!debug.contains(value), "Debug leaked {value:?}: {debug}"); + } + } + + #[test] + fn all_auth_sources_are_excluded_from_the_final_identity() { + let dbms_a = InfobaseDbmsConfig::new("PostgreSQL", "db", "accounting") + .with_credentials(Some("db-user-a".to_owned()), Some("db-pass-a".to_owned())); + let dbms_b = InfobaseDbmsConfig::new("PostgreSQL", "db", "accounting") + .with_credentials(Some("db-user-b".to_owned()), Some("db-pass-b".to_owned())); + let plain = InfobaseConfig { + connection: "Srvr=demo;Ref=accounting;Usr=plain-user;Pwd=plain-pass".to_owned(), + user: Some("config-user-a".to_owned()), + password: Some("config-pass-a".to_owned()), + dbms: Some(dbms_a), + }; + let raw = InfobaseConfig { + connection: "/S demo\\accounting /N raw-user /P raw-pass".to_owned(), + user: Some("config-user-b".to_owned()), + password: Some("config-pass-b".to_owned()), + dbms: Some(dbms_b), + }; + + assert_eq!( + InfobaseIdentity::normalize(&plain) + .expect("plain identity") + .fingerprint(), + InfobaseIdentity::normalize(&raw) + .expect("raw identity") + .fingerprint() + ); + } + + #[test] + fn unsupported_raw_connection_is_rejected_without_echoing_it() { + let error = InfobaseIdentity::normalize(&file_infobase("/Unknown secret-value")) + .expect_err("unsupported raw form"); + + assert!(!error.to_string().contains("secret-value")); + } + + #[test] + fn distinct_infobases_and_dbms_targets_have_distinct_identities() { + let dir = tempdir().expect("tempdir"); + let first = InfobaseIdentity::normalize(&file_infobase(format!( + "File={}", + dir.path().join("first").display() + ))) + .expect("first"); + let second = InfobaseIdentity::normalize(&file_infobase(format!( + "File={}", + dir.path().join("second").display() + ))) + .expect("second"); + assert_ne!(first.fingerprint(), second.fingerprint()); + + let server_a = InfobaseConfig::server( + "Srvr=cluster;Ref=accounting", + InfobaseDbmsConfig::new("PostgreSQL", "db-a", "accounting"), + ); + let server_b = InfobaseConfig::server( + "Srvr=cluster;Ref=accounting", + InfobaseDbmsConfig::new("PostgreSQL", "db-b", "accounting"), + ); + assert_ne!( + InfobaseIdentity::normalize(&server_a) + .expect("server a") + .fingerprint(), + InfobaseIdentity::normalize(&server_b) + .expect("server b") + .fingerprint() + ); + } + + #[test] + fn layout_is_versioned_and_context_identity_covers_every_contract_field() { + let dir = tempdir().expect("tempdir"); + let work = dir.path().join("work"); + let source = dir.path().join("source"); + std::fs::create_dir(&work).expect("work"); + std::fs::create_dir(&source).expect("source"); + let identity = InfobaseIdentity::normalize(&file_infobase(format!( + "File={}", + dir.path().join("ib").display() + ))) + .expect("identity"); + let ib_fingerprint = identity.fingerprint().to_owned(); + let layout = RuntimeStateLayout::new(&work, identity).expect("layout"); + let descriptor = RuntimeSourceDescriptor::new(RuntimeSourceIdentityInputs { + configured_source_identity: Path::new("configured/source"), + source_root: &source, + purpose: SourceSetPurpose::Configuration, + format: SourceFormat::Designer, + backend: BuilderBackend::Designer, + logical_role: LogicalSourceRole::DesignerSource, + }) + .expect("descriptor"); + let state = layout.source_state("main", &descriptor); + + assert_eq!( + state.state_dir(), + std::fs::canonicalize(&work) + .expect("canonical work") + .join("ib-state") + .join("v1") + .join(ib_fingerprint) + .join(format!("main-{}", state.context_fingerprint())) + ); + assert_eq!( + state.hash_storage_path(), + state.state_dir().join("hash-storage.redb") + ); + assert_eq!( + state.private_cdfi_path(), + state.state_dir().join("ConfigDumpInfo.xml") + ); + assert_eq!( + state.transactions_dir(), + state.state_dir().join("transactions") + ); + assert_eq!( + state + .ib_baseline(StateGeneration::new(7)) + .generation() + .value(), + 7 + ); + assert_eq!( + state + .source_observation(StateGeneration::new(7)) + .generation() + .value(), + 7 + ); + + let variants = [ + RuntimeSourceDescriptor::new(RuntimeSourceIdentityInputs { + configured_source_identity: Path::new("configured/other"), + source_root: &source, + purpose: SourceSetPurpose::Configuration, + format: SourceFormat::Designer, + backend: BuilderBackend::Designer, + logical_role: LogicalSourceRole::DesignerSource, + }) + .expect("raw identity"), + RuntimeSourceDescriptor::new(RuntimeSourceIdentityInputs { + configured_source_identity: Path::new("configured/source"), + source_root: &dir.path().join("other-root"), + purpose: SourceSetPurpose::Configuration, + format: SourceFormat::Designer, + backend: BuilderBackend::Designer, + logical_role: LogicalSourceRole::DesignerSource, + }) + .expect("root identity"), + RuntimeSourceDescriptor::new(RuntimeSourceIdentityInputs { + configured_source_identity: Path::new("configured/source"), + source_root: &source, + purpose: SourceSetPurpose::Extension, + format: SourceFormat::Designer, + backend: BuilderBackend::Designer, + logical_role: LogicalSourceRole::DesignerSource, + }) + .expect("purpose identity"), + RuntimeSourceDescriptor::new(RuntimeSourceIdentityInputs { + configured_source_identity: Path::new("configured/source"), + source_root: &source, + purpose: SourceSetPurpose::Configuration, + format: SourceFormat::Edt, + backend: BuilderBackend::Designer, + logical_role: LogicalSourceRole::DesignerSource, + }) + .expect("format identity"), + RuntimeSourceDescriptor::new(RuntimeSourceIdentityInputs { + configured_source_identity: Path::new("configured/source"), + source_root: &source, + purpose: SourceSetPurpose::Configuration, + format: SourceFormat::Designer, + backend: BuilderBackend::Ibcmd, + logical_role: LogicalSourceRole::DesignerSource, + }) + .expect("backend identity"), + RuntimeSourceDescriptor::new(RuntimeSourceIdentityInputs { + configured_source_identity: Path::new("configured/source"), + source_root: &source, + purpose: SourceSetPurpose::Configuration, + format: SourceFormat::Designer, + backend: BuilderBackend::Designer, + logical_role: LogicalSourceRole::EdtSource, + }) + .expect("role identity"), + ]; + for variant in variants { + assert_ne!(descriptor.fingerprint(), variant.fingerprint()); + } + } + + #[test] + fn source_set_name_is_sanitized_to_one_safe_segment() { + let dir = tempdir().expect("tempdir"); + let layout = RuntimeStateLayout::new( + dir.path().join("work"), + InfobaseIdentity::normalize(&file_infobase(format!( + "File={}", + dir.path().join("ib").display() + ))) + .expect("identity"), + ) + .expect("layout"); + let descriptor = RuntimeSourceDescriptor::new(RuntimeSourceIdentityInputs { + configured_source_identity: Path::new("source"), + source_root: &dir.path().join("source"), + purpose: SourceSetPurpose::Configuration, + format: SourceFormat::Designer, + backend: BuilderBackend::Designer, + logical_role: LogicalSourceRole::DesignerSource, + }) + .expect("descriptor"); + + let state = layout.source_state("../unsafe/name", &descriptor); + let segment = state + .state_dir() + .file_name() + .and_then(|name| name.to_str()) + .expect("state segment"); + assert!(!segment.contains('/')); + assert!(!segment.contains("..")); + } + + #[test] + fn sanitized_source_set_collisions_still_have_distinct_context_fingerprints() { + let dir = tempdir().expect("tempdir"); + let layout = RuntimeStateLayout::new( + dir.path().join("work"), + InfobaseIdentity::normalize(&file_infobase(format!( + "File={}", + dir.path().join("ib").display() + ))) + .expect("identity"), + ) + .expect("layout"); + let descriptor = RuntimeSourceDescriptor::new(RuntimeSourceIdentityInputs { + configured_source_identity: Path::new("source"), + source_root: &dir.path().join("source"), + purpose: SourceSetPurpose::Configuration, + format: SourceFormat::Designer, + backend: BuilderBackend::Designer, + logical_role: LogicalSourceRole::DesignerSource, + }) + .expect("descriptor"); + + let slash = layout.source_state("unsafe/name", &descriptor); + let backslash = layout.source_state("unsafe\\name", &descriptor); + + assert_ne!(slash.state_dir(), backslash.state_dir()); + } + + #[cfg(unix)] + #[test] + fn canonical_path_preserves_symlink_parent_semantics() { + use std::os::unix::fs::symlink; + + let dir = tempdir().expect("tempdir"); + let real = dir.path().join("real"); + let child = real.join("child"); + let target = real.join("target"); + let link = dir.path().join("link"); + std::fs::create_dir_all(&child).expect("child"); + std::fs::create_dir(&target).expect("target"); + symlink(&child, &link).expect("link"); + + assert_eq!( + InfobaseIdentity::normalize(&file_infobase(format!( + "File={}", + link.join("..").join("target").display() + ))) + .expect("symlink parent identity") + .fingerprint(), + InfobaseIdentity::normalize(&file_infobase(format!("File={}", target.display()))) + .expect("target identity") + .fingerprint() + ); + } + + #[test] + fn baseline_and_observation_are_distinct_generation_bound_handles() { + fn baseline_path(value: &super::IbBaseline) -> PathBuf { + value.path().to_path_buf() + } + fn observation_path(value: &super::SourceObservation) -> PathBuf { + value.path().to_path_buf() + } + + let dir = tempdir().expect("tempdir"); + let layout = RuntimeStateLayout::new( + dir.path().join("work"), + InfobaseIdentity::normalize(&file_infobase(format!( + "File={}", + dir.path().join("ib").display() + ))) + .expect("identity"), + ) + .expect("layout"); + let descriptor = RuntimeSourceDescriptor::new(RuntimeSourceIdentityInputs { + configured_source_identity: Path::new("source"), + source_root: &dir.path().join("source"), + purpose: SourceSetPurpose::Configuration, + format: SourceFormat::Designer, + backend: BuilderBackend::Designer, + logical_role: LogicalSourceRole::DesignerSource, + }) + .expect("descriptor"); + let state = layout.source_state("main", &descriptor); + let baseline = state.ib_baseline(StateGeneration::new(3)); + let observation = state.source_observation(StateGeneration::new(3)); + + assert_ne!(baseline_path(&baseline), observation_path(&observation)); + assert!(baseline.path().starts_with(state.state_dir())); + assert!(observation.path().starts_with(state.state_dir())); + } + + #[test] + fn baseline_roles_are_distinct_within_one_generation() { + let dir = tempdir().expect("tempdir"); + let layout = RuntimeStateLayout::new( + dir.path().join("work"), + InfobaseIdentity::normalize(&file_infobase(format!( + "File={}", + dir.path().join("ib").display() + ))) + .expect("identity"), + ) + .expect("layout"); + let descriptor = RuntimeSourceDescriptor::new(RuntimeSourceIdentityInputs { + configured_source_identity: Path::new("source"), + source_root: &dir.path().join("source"), + purpose: SourceSetPurpose::Configuration, + format: SourceFormat::Edt, + backend: BuilderBackend::Designer, + logical_role: LogicalSourceRole::EdtSource, + }) + .expect("descriptor"); + let state = layout.source_state("main", &descriptor); + let generation = StateGeneration::new(5); + let configured = state.baseline(BaselineRole::ConfiguredSource, generation); + let platform = state.baseline(BaselineRole::EdtPlatformDesigner, generation); + + assert_eq!(configured.role(), BaselineRole::ConfiguredSource); + assert_eq!(platform.role(), BaselineRole::EdtPlatformDesigner); + assert_ne!(configured.path(), platform.path()); + assert_eq!(configured.generation(), platform.generation()); + } +} diff --git a/src/domain/source_set.rs b/src/domain/source_set.rs index 640bad2..4762ced 100644 --- a/src/domain/source_set.rs +++ b/src/domain/source_set.rs @@ -1,6 +1,6 @@ use std::path::{Path, PathBuf}; -use crate::support::path::is_safe_path_segment; +use crate::domain::runtime_state::{BaselineRole, IbBaseline, RuntimeSourceState, StateGeneration}; /// Runtime context for one logical source-set. #[derive(Debug, Clone)] @@ -9,31 +9,32 @@ pub struct SourceSetContext { name: String, /// Absolute root directory of the sources. path: PathBuf, - /// Key used to name the redb hash-storage file (`workPath/hash-storages/.redb`). - storage_key: String, + /// Already resolved, versioned runtime state for this source view. + runtime_state: RuntimeSourceState, + excluded_roots: Vec, } impl SourceSetContext { - pub fn new(name: impl Into, path: PathBuf, storage_key: impl Into) -> Self { - let name = name.into(); - let storage_key = storage_key.into(); + pub fn new(name: impl Into, path: PathBuf, runtime_state: RuntimeSourceState) -> Self { assert!( path.is_absolute(), "SourceSetContext.path must be absolute, got: {}", path.display() ); - assert!( - is_safe_path_segment(&storage_key), - "SourceSetContext.storage_key must be a safe single path segment, got: {storage_key}" - ); Self { - name, + name: name.into(), path, - storage_key, + runtime_state, + excluded_roots: Vec::new(), } } + pub fn with_excluded_roots(mut self, excluded_roots: Vec) -> Self { + self.excluded_roots = excluded_roots; + self + } + pub fn name(&self) -> &str { &self.name } @@ -42,62 +43,109 @@ impl SourceSetContext { &self.path } - /// Absolute path to the redb hash-storage file for this context. - pub fn storage_path(&self, work_path: &Path) -> PathBuf { - work_path - .join("hash-storages") - .join(format!("{}.redb", self.storage_key)) + pub fn excluded_roots(&self) -> &[PathBuf] { + &self.excluded_roots + } + + #[cfg(test)] + pub fn runtime_state(&self) -> &RuntimeSourceState { + &self.runtime_state + } + + /// Absolute path to this context's versioned redb hash storage. + pub fn storage_path(&self) -> PathBuf { + self.runtime_state.hash_storage_path() + } + + /// Private platform-owned `ConfigDumpInfo.xml` for this source view. + pub(crate) fn private_cdfi_path(&self) -> PathBuf { + self.runtime_state.private_cdfi_path() + } + + /// Complete private baseline for a semantic role and state generation. + pub(crate) fn baseline(&self, role: BaselineRole, generation: StateGeneration) -> IbBaseline { + self.runtime_state.baseline(role, generation) + } + + /// Owned transaction directory for this source view. + pub(crate) fn transactions_dir(&self) -> PathBuf { + self.runtime_state.transactions_dir() + } + + /// Per-source lock serializing recovery, staging and runtime-state publication. + pub(crate) fn state_lock_path(&self) -> PathBuf { + self.transactions_dir() + .parent() + .map(|state_dir| state_dir.join("runtime-state.lock")) + .unwrap_or_else(|| self.transactions_dir().join("runtime-state.lock")) } } #[cfg(test)] mod tests { use super::SourceSetContext; - use std::path::PathBuf; + use crate::config::model::{BuilderBackend, InfobaseConfig, SourceFormat, SourceSetPurpose}; + use crate::domain::runtime_state::{ + InfobaseIdentity, LogicalSourceRole, RuntimeSourceDescriptor, RuntimeSourceIdentityInputs, + RuntimeStateLayout, + }; + use std::path::{Path, PathBuf}; + use tempfile::tempdir; + + fn context(path: PathBuf) -> SourceSetContext { + let dir = tempdir().expect("tempdir"); + let identity = InfobaseIdentity::normalize(&InfobaseConfig::file(format!( + "File={}", + dir.path().join("ib").display() + ))) + .expect("identity"); + let layout = RuntimeStateLayout::new(dir.path().join("work"), identity).expect("layout"); + let descriptor = RuntimeSourceDescriptor::new(RuntimeSourceIdentityInputs { + configured_source_identity: Path::new("src-main"), + source_root: &path, + purpose: SourceSetPurpose::Configuration, + format: SourceFormat::Designer, + backend: BuilderBackend::Designer, + logical_role: LogicalSourceRole::DesignerSource, + }) + .expect("descriptor"); + let state = layout.source_state("main", &descriptor); + SourceSetContext::new("main", path, state) + } #[test] - fn accepts_absolute_path() { - let context = - SourceSetContext::new("main", PathBuf::from("/tmp/src-main"), "designer-main"); + fn uses_resolved_versioned_storage_path() { + let source = std::env::current_dir() + .expect("current dir") + .join("target/source-set-context"); + let context = context(source.clone()); + assert_eq!(context.name(), "main"); - assert_eq!(context.path(), PathBuf::from("/tmp/src-main").as_path()); + assert_eq!(context.path(), source); assert_eq!( - context.storage_path(PathBuf::from("/tmp/work").as_path()), - PathBuf::from("/tmp/work/hash-storages/designer-main.redb") + context.storage_path(), + context + .runtime_state() + .state_dir() + .join("hash-storage.redb") ); + assert!(!context + .storage_path() + .to_string_lossy() + .contains("hash-storages")); } #[test] #[should_panic(expected = "must be absolute")] fn rejects_relative_path() { - let _ = SourceSetContext::new("main", PathBuf::from("relative/path"), "designer-main"); - } - - #[test] - fn accepts_safe_storage_key() { - let context = - SourceSetContext::new("main", PathBuf::from("/tmp/src-main"), "main-config_01"); - assert_eq!( - context.storage_path(PathBuf::from("/tmp/work").as_path()), - PathBuf::from("/tmp/work/hash-storages/main-config_01.redb") + let absolute = std::env::current_dir() + .expect("current dir") + .join("target/source-set-context"); + let valid = context(absolute); + let _ = SourceSetContext::new( + "main", + PathBuf::from("relative/path"), + valid.runtime_state().clone(), ); } - - #[test] - #[should_panic(expected = "safe single path segment")] - fn rejects_storage_key_with_parent_traversal() { - let _ = SourceSetContext::new("main", PathBuf::from("/tmp/src-main"), "../outside"); - } - - #[test] - #[should_panic(expected = "safe single path segment")] - fn rejects_storage_key_with_separator() { - let _ = SourceSetContext::new("main", PathBuf::from("/tmp/src-main"), "bad/name"); - } - - #[test] - #[should_panic(expected = "safe single path segment")] - fn rejects_storage_key_with_backslash_separator() { - let _ = SourceSetContext::new("main", PathBuf::from("/tmp/src-main"), "bad\\name"); - } } diff --git a/src/domain/sync_receipt.rs b/src/domain/sync_receipt.rs new file mode 100644 index 0000000..631758e --- /dev/null +++ b/src/domain/sync_receipt.rs @@ -0,0 +1,409 @@ +use serde::{Deserialize, Serialize}; +use std::collections::{BTreeMap, BTreeSet}; +use std::path::{Component, Path}; +use thiserror::Error; + +#[derive(Debug, Clone, Error, PartialEq, Eq)] +pub enum SyncReceiptError { + #[error("sync target path must be a normalized relative path: '{0}'")] + InvalidPath(String), + #[error("sync target hash must be 64 lowercase hexadecimal characters: '{0}'")] + InvalidHash(String), + #[error("sync target '{0}' must contain a preHash or postHash")] + MissingHashes(String), + #[error("sync receipt list '{list}' contains contradictory entries for '{path}'")] + ContradictoryDuplicate { list: &'static str, path: String }, + #[error("sync receipt outcome lists overlap at '{0}'")] + OverlappingOutcome(String), + #[error("sync receipt contains inconsistent hashes for '{0}'")] + InconsistentTarget(String), + #[error("sync receipt violates terminal status invariants for '{0:?}'")] + InvalidTerminalState(SyncStatus), +} + +#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)] +#[serde(rename_all = "lowercase")] +pub enum SyncStatus { + Applied, + Skipped, + Failed, + Conflict, +} + +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, PartialOrd, Ord)] +#[serde(rename_all = "camelCase", try_from = "SyncTargetDto")] +pub struct SyncTarget { + path: String, + pre_hash: Option, + post_hash: Option, +} + +#[derive(Deserialize)] +#[serde(rename_all = "camelCase")] +struct SyncTargetDto { + path: String, + pre_hash: Option, + post_hash: Option, +} + +impl SyncTarget { + pub fn new( + path: impl AsRef, + pre_hash: Option, + post_hash: Option, + ) -> Result { + let path = path.as_ref(); + let windows_absolute = path + .as_bytes() + .get(1) + .is_some_and(|separator| *separator == b':'); + let normalized_segments = !path.is_empty() + && !path.starts_with('/') + && !path.contains('\\') + && !windows_absolute + && path + .split('/') + .all(|segment| !segment.is_empty() && segment != "." && segment != ".."); + let normal_components = Path::new(path) + .components() + .all(|component| matches!(component, Component::Normal(_))); + if !normalized_segments || !normal_components { + return Err(SyncReceiptError::InvalidPath(path.to_owned())); + } + if pre_hash.is_none() && post_hash.is_none() { + return Err(SyncReceiptError::MissingHashes(path.to_owned())); + } + for hash in pre_hash.iter().chain(post_hash.iter()) { + if hash.len() != 64 + || !hash + .bytes() + .all(|byte| byte.is_ascii_digit() || (b'a'..=b'f').contains(&byte)) + { + return Err(SyncReceiptError::InvalidHash(hash.clone())); + } + } + Ok(Self { + path: path.to_owned(), + pre_hash, + post_hash, + }) + } + + pub fn path(&self) -> &str { + &self.path + } + + pub fn pre_hash(&self) -> Option<&str> { + self.pre_hash.as_deref() + } + + pub fn post_hash(&self) -> Option<&str> { + self.post_hash.as_deref() + } +} + +impl TryFrom for SyncTarget { + type Error = SyncReceiptError; + + fn try_from(value: SyncTargetDto) -> Result { + Self::new(value.path, value.pre_hash, value.post_hash) + } +} + +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] +#[serde(rename_all = "camelCase", try_from = "SyncReceiptDto")] +pub struct SyncReceipt { + status: SyncStatus, + requested: Vec, + processed: Vec, + skipped: Vec, + conflicted: Vec, +} + +#[derive(Deserialize)] +#[serde(rename_all = "camelCase")] +struct SyncReceiptDto { + status: SyncStatus, + requested: Vec, + processed: Vec, + skipped: Vec, + conflicted: Vec, +} + +impl SyncReceipt { + pub fn applied( + requested: Vec, + processed: Vec, + skipped: Vec, + ) -> Result { + Self::validated(SyncStatus::Applied, requested, processed, skipped, vec![]) + } + + pub fn skipped( + requested: Vec, + skipped: Vec, + ) -> Result { + Self::validated(SyncStatus::Skipped, requested, vec![], skipped, vec![]) + } + + pub fn failed(requested: Vec) -> Result { + Self::validated(SyncStatus::Failed, requested, vec![], vec![], vec![]) + } + + pub fn conflict( + requested: Vec, + skipped: Vec, + conflicted: Vec, + ) -> Result { + Self::validated(SyncStatus::Conflict, requested, vec![], skipped, conflicted) + } + + pub fn empty_skipped() -> Self { + Self::empty(SyncStatus::Skipped) + } + + pub fn empty_applied() -> Self { + Self::empty(SyncStatus::Applied) + } + + pub fn empty_failed() -> Self { + Self::empty(SyncStatus::Failed) + } + + fn empty(status: SyncStatus) -> Self { + Self { + status, + requested: vec![], + processed: vec![], + skipped: vec![], + conflicted: vec![], + } + } + + fn validated( + status: SyncStatus, + requested: Vec, + processed: Vec, + skipped: Vec, + conflicted: Vec, + ) -> Result { + let requested = unique("requested", requested)?; + let processed = unique("processed", processed)?; + let skipped = unique("skipped", skipped)?; + let conflicted = unique("conflicted", conflicted)?; + + let processed_by_path = processed + .iter() + .map(|target| (target.path(), target)) + .collect::>(); + for target in &skipped { + if processed_by_path + .get(target.path()) + .is_some_and(|processed| status != SyncStatus::Applied || *processed != target) + { + return Err(SyncReceiptError::OverlappingOutcome(target.path.clone())); + } + } + let conflicted_paths = conflicted + .iter() + .map(SyncTarget::path) + .collect::>(); + for target in processed.iter().chain(skipped.iter()) { + if conflicted_paths.contains(target.path()) { + return Err(SyncReceiptError::OverlappingOutcome(target.path.clone())); + } + } + let requested_by_path = requested + .iter() + .map(|target| (target.path(), target)) + .collect::>(); + for target in processed + .iter() + .chain(skipped.iter()) + .chain(conflicted.iter()) + { + if requested_by_path + .get(target.path()) + .is_some_and(|requested| *requested != target) + { + return Err(SyncReceiptError::InconsistentTarget(target.path.clone())); + } + } + + Ok(Self { + status, + requested, + processed, + skipped, + conflicted, + }) + } +} + +fn unique( + list: &'static str, + mut targets: Vec, +) -> Result, SyncReceiptError> { + targets.sort_by(|left, right| left.path.cmp(&right.path)); + for pair in targets.windows(2) { + if pair[0].path == pair[1].path { + return Err(SyncReceiptError::ContradictoryDuplicate { + list, + path: pair[0].path.clone(), + }); + } + } + Ok(targets) +} + +impl Default for SyncReceipt { + fn default() -> Self { + Self::empty_skipped() + } +} + +impl TryFrom for SyncReceipt { + type Error = SyncReceiptError; + + fn try_from(value: SyncReceiptDto) -> Result { + let valid = match value.status { + SyncStatus::Applied => value.conflicted.is_empty(), + SyncStatus::Skipped => value.processed.is_empty() && value.conflicted.is_empty(), + SyncStatus::Failed => { + value.processed.is_empty() + && value.skipped.is_empty() + && value.conflicted.is_empty() + } + SyncStatus::Conflict => value.processed.is_empty(), + }; + if !valid { + return Err(SyncReceiptError::InvalidTerminalState(value.status)); + } + match value.status { + SyncStatus::Applied => Self::applied(value.requested, value.processed, value.skipped), + SyncStatus::Skipped => Self::skipped(value.requested, value.skipped), + SyncStatus::Failed => Self::failed(value.requested), + SyncStatus::Conflict => { + Self::conflict(value.requested, value.skipped, value.conflicted) + } + } + } +} + +#[cfg(test)] +mod tests { + use super::{SyncReceipt, SyncTarget}; + + const PRE: &str = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; + const POST: &str = "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"; + + fn target(path: &str) -> SyncTarget { + SyncTarget::new(path, Some(PRE.to_owned()), Some(POST.to_owned())).expect("target") + } + + #[test] + fn rejects_invalid_paths_hashes_and_empty_delta() { + for path in [ + "", + "/absolute.xml", + "C:/absolute.xml", + "a/../b.xml", + "a//b", + "a\\b", + ] { + assert!( + SyncTarget::new(path, Some(PRE.to_owned()), None).is_err(), + "accepted {path}" + ); + } + assert!(SyncTarget::new("a.xml", None, None).is_err()); + for hash in [ + "pre", + "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", + "gggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggg", + ] { + assert!(SyncTarget::new("a.xml", Some(hash.to_owned()), None).is_err()); + } + } + + #[test] + fn constructors_sort_and_reject_duplicate_or_incoherent_membership() { + let receipt = SyncReceipt::applied( + vec![target("z.xml"), target("a.xml")], + vec![target("z.xml")], + vec![], + ) + .expect("receipt"); + let value = serde_json::to_value(receipt).expect("serialize"); + assert_eq!(value["requested"][0]["path"], "a.xml"); + assert_eq!(value["requested"][1]["path"], "z.xml"); + + assert!( + SyncReceipt::applied(vec![target("a.xml"), target("a.xml")], vec![], vec![]).is_err() + ); + let changed = SyncTarget::new("a.xml", Some(PRE.to_owned()), None).expect("changed"); + assert!(SyncReceipt::applied(vec![target("a.xml")], vec![changed], vec![]).is_err()); + let orthogonal = SyncReceipt::applied( + vec![target("a.xml")], + vec![target("a.xml")], + vec![target("a.xml")], + ) + .expect("processed and retained-local evidence may overlap"); + let json = serde_json::to_value(&orthogonal).expect("serialize overlap"); + assert_eq!(json["processed"][0], json["skipped"][0]); + assert_eq!( + serde_json::from_value::(json).expect("round trip overlap"), + orthogonal + ); + + let changed = SyncTarget::new("a.xml", Some(PRE.to_owned()), None).expect("changed"); + assert!(SyncReceipt::applied(vec![], vec![target("a.xml")], vec![changed]).is_err()); + } + + #[test] + fn conflict_round_trip_preserves_nonempty_skipped_and_conflicted() { + let receipt = SyncReceipt::conflict( + vec![target("skipped.xml"), target("conflict.xml")], + vec![target("skipped.xml")], + vec![target("conflict.xml")], + ) + .expect("receipt"); + let json = serde_json::to_value(&receipt).expect("serialize"); + assert_eq!(json["processed"], serde_json::json!([])); + assert_eq!(json["skipped"][0]["path"], "skipped.xml"); + assert_eq!(json["conflicted"][0]["path"], "conflict.xml"); + assert_eq!( + serde_json::from_value::(json).expect("round trip"), + receipt + ); + } + + #[test] + fn json_is_camel_case_and_rejects_invalid_terminal_state() { + let receipt = SyncReceipt::applied(vec![], vec![target("a.xml")], vec![]).expect("receipt"); + let value = serde_json::to_value(receipt).expect("serialize"); + assert_eq!(value["processed"][0]["preHash"], PRE); + assert_eq!(value["processed"][0]["postHash"], POST); + + let invalid = serde_json::json!({ + "status": "failed", + "requested": [], + "processed": [{"path":"a.xml","preHash":null,"postHash":POST}], + "skipped": [], + "conflicted": [] + }); + assert!(serde_json::from_value::(invalid).is_err()); + + let duplicate = serde_json::json!({ + "status": "failed", + "requested": [ + {"path":"a.xml","preHash":null,"postHash":POST}, + {"path":"a.xml","preHash":PRE,"postHash":POST} + ], + "processed": [], + "skipped": [], + "conflicted": [] + }); + assert!(serde_json::from_value::(duplicate).is_err()); + } +} diff --git a/src/mcp/edt_syntax.rs b/src/mcp/edt_syntax.rs index 23aad7d..249cb35 100644 --- a/src/mcp/edt_syntax.rs +++ b/src/mcp/edt_syntax.rs @@ -73,7 +73,26 @@ pub async fn execute( ))); } - let inventory = SourceSetInventory::new(config); + let inventory = match SourceSetInventory::new(config) { + Ok(inventory) => inventory, + Err(error) => { + let error = AppError::from(error); + let error_message = error.to_string(); + return Ok(Err(SyntaxExecutionFailure::with_payload( + error, + failed_result( + "edt", + SyntaxCheckStatus::ToolFailed, + -1, + started, + vec![], + None, + Some(error_message), + None, + ), + ))); + } + }; let source_sets = match resolve_edt_source_sets(&inventory, projects) { Ok(source_sets) => source_sets, Err(error) => { diff --git a/src/mcp/service.rs b/src/mcp/service.rs index 66cc806..4eb0a4e 100644 --- a/src/mcp/service.rs +++ b/src/mcp/service.rs @@ -1157,6 +1157,7 @@ mod tests { ok: true, message: Some("loaded".to_owned()), duration_ms: 17, + receipt: crate::domain::sync_receipt::SyncReceipt::empty_applied(), }], duration_ms: 42, })); @@ -1178,6 +1179,7 @@ mod tests { assert_eq!(response.duration_ms, 42); assert_eq!(response.data["ok"], true); assert_eq!(response.data["steps"][0]["mode"], "full"); + assert_eq!(response.data["steps"][0]["receipt"]["status"], "applied"); let requests = service.port.build_requests.borrow(); assert_eq!(requests.len(), 1); assert_eq!(requests[0].0.command(), CommandName::Build); @@ -1198,6 +1200,7 @@ mod tests { ok: false, message: Some("broken".to_owned()), duration_ms: 9, + receipt: crate::domain::sync_receipt::SyncReceipt::empty_failed(), }], duration_ms: 19, }, @@ -1216,6 +1219,10 @@ mod tests { assert_eq!(failure.response.command, "build"); assert_eq!(failure.response.duration_ms, 19); assert_eq!(failure.response.data["steps"][0]["ok"], false); + assert_eq!( + failure.response.data["steps"][0]["receipt"]["status"], + "failed" + ); assert_eq!( failure .response @@ -1487,6 +1494,7 @@ mod tests { platform_log_path: None, duration_ms: 33, message: None, + receipt: crate::domain::sync_receipt::SyncReceipt::empty_applied(), })); let config = sample_config(); let service = McpService::with_port(&config, port); @@ -1505,6 +1513,7 @@ mod tests { assert!(response.ok); assert_eq!(response.command, "dump"); assert_eq!(response.data["mode"], "INCREMENTAL"); + assert_eq!(response.data["receipt"]["status"], "applied"); let requests = service.port.dump_requests.borrow(); assert_eq!(requests[0].1.mode, DumpModeRequest::Incremental); assert_eq!(requests[0].1.extension, None); @@ -1528,6 +1537,7 @@ mod tests { platform_log_path: None, duration_ms: 3, message: Some("dump failed".to_owned()), + receipt: crate::domain::sync_receipt::SyncReceipt::empty_failed(), }, ))), ); @@ -1550,6 +1560,7 @@ mod tests { assert_eq!(failure.response.command, "dump"); assert_eq!(failure.response.data["mode"], "INCREMENTAL"); assert_eq!(failure.response.data["message"], "dump failed"); + assert_eq!(failure.response.data["receipt"]["status"], "failed"); } other => panic!("unexpected error: {other:?}"), } @@ -1605,6 +1616,7 @@ mod tests { platform_log_path: None, duration_ms: 1, message: None, + receipt: Default::default(), })), ); @@ -1656,6 +1668,7 @@ mod tests { "IBCMD does not support object-scoped partial dump; ran incremental export for source-set 'main' instead" .to_owned(), ), + receipt: Default::default(), })); let config = sample_config(); let service = McpService::with_port(&config, port); @@ -1705,6 +1718,7 @@ mod tests { "IBCMD does not support object-scoped partial dump; export failed" .to_owned(), ), + receipt: Default::default(), }, ))), ); diff --git a/src/platform/connection.rs b/src/platform/connection.rs index c4880fa..0e67b6b 100644 --- a/src/platform/connection.rs +++ b/src/platform/connection.rs @@ -1,3 +1,5 @@ +use crate::support::connection_args::split_v8_arg_string; + /// Parsed V8 connection and optional authentication parameters. #[derive(Debug, Clone)] pub struct V8Connection { @@ -14,7 +16,7 @@ impl V8Connection { pub fn from_connection_string(raw: &str) -> Self { let trimmed = raw.trim(); let connection_args = if trimmed.starts_with('/') || trimmed.starts_with('-') { - split_arg_string(trimmed) + split_v8_arg_string(trimmed).0 } else { vec!["/IBConnectionString".to_owned(), trimmed.to_owned()] }; @@ -78,30 +80,6 @@ fn file_path_from_args(args: &[String]) -> Option<&str> { None } -fn split_arg_string(raw: &str) -> Vec { - let mut args = Vec::new(); - let mut current = String::new(); - let mut in_quotes = false; - - for ch in raw.chars() { - match ch { - '"' => in_quotes = !in_quotes, - ch if ch.is_whitespace() && !in_quotes => { - if !current.is_empty() { - args.push(std::mem::take(&mut current)); - } - } - _ => current.push(ch), - } - } - - if !current.is_empty() { - args.push(current); - } - - args -} - #[cfg(test)] mod tests { use super::V8Connection; diff --git a/src/platform/designer.rs b/src/platform/designer.rs index 7cf348b..1c5e208 100644 --- a/src/platform/designer.rs +++ b/src/platform/designer.rs @@ -179,7 +179,7 @@ impl<'a> DesignerDsl<'a> { self.run(&args) } - /// `/DumpConfigToFiles [-Extension ]` + /// `/DumpConfigToFiles -updateConfigDumpInfo [-Extension ]` pub fn dump_config_to_files( &self, target_dir: &Path, @@ -188,6 +188,7 @@ impl<'a> DesignerDsl<'a> { let mut args = self.base_args(); args.push("/DumpConfigToFiles".to_owned()); args.push(target_dir.display().to_string()); + args.push("-updateConfigDumpInfo".to_owned()); if let Some(extension) = extension { args.push("-Extension".to_owned()); args.push(extension.to_owned()); @@ -195,7 +196,7 @@ impl<'a> DesignerDsl<'a> { self.run(&args) } - /// `/DumpConfigToFiles -update [-Extension ]` + /// `/DumpConfigToFiles -update -updateConfigDumpInfo [-Extension ]` pub fn dump_config_to_files_incremental( &self, target_dir: &Path, @@ -205,6 +206,7 @@ impl<'a> DesignerDsl<'a> { args.push("/DumpConfigToFiles".to_owned()); args.push(target_dir.display().to_string()); args.push("-update".to_owned()); + args.push("-updateConfigDumpInfo".to_owned()); if let Some(extension) = extension { args.push("-Extension".to_owned()); args.push(extension.to_owned()); @@ -254,7 +256,8 @@ impl<'a> DesignerDsl<'a> { self.run(&args) } - /// `/DumpConfigToFiles -partial -listFile [-Extension ]` + /// `/DumpConfigToFiles -partial -listFile -updateConfigDumpInfo + /// [-Extension ]` pub fn dump_config_to_files_partial( &self, target_dir: &Path, @@ -267,6 +270,7 @@ impl<'a> DesignerDsl<'a> { args.push("-partial".to_owned()); args.push("-listFile".to_owned()); args.push(list_file.display().to_string()); + args.push("-updateConfigDumpInfo".to_owned()); if let Some(extension) = extension { args.push("-Extension".to_owned()); args.push(extension.to_owned()); @@ -476,8 +480,8 @@ mod tests { let args = fs::read_to_string(args_log).expect("args log"); assert!(args.contains("/DumpConfigToFiles")); - assert!(!args.contains("-update")); - assert!(!args.contains("-updateConfigDumpInfo")); + assert!(!args.lines().any(|arg| arg == "-update")); + assert!(args.contains("-updateConfigDumpInfo")); } #[cfg(unix)] @@ -504,7 +508,7 @@ mod tests { let args = fs::read_to_string(args_log).expect("args log"); assert!(args.contains("/DumpConfigToFiles")); assert!(args.contains("-update")); - assert!(!args.contains("-updateConfigDumpInfo")); + assert!(args.contains("-updateConfigDumpInfo")); assert!(args.contains("-Extension")); assert!(args.contains("ExtName")); } @@ -539,7 +543,7 @@ mod tests { assert!(args.contains("-partial")); assert!(args.contains("-listFile")); assert!(args.contains("objects.txt")); - assert!(!args.contains("-updateConfigDumpInfo")); + assert!(args.contains("-updateConfigDumpInfo")); assert!(args.contains("-Extension")); assert!(args.contains("ExtName")); } diff --git a/src/support/connection_args.rs b/src/support/connection_args.rs new file mode 100644 index 0000000..a17dac6 --- /dev/null +++ b/src/support/connection_args.rs @@ -0,0 +1,27 @@ +/// Split a raw V8 connection argument string using the same quote rules as process execution. +/// +/// Backslashes are ordinary characters. The boolean reports whether all double quotes were +/// balanced so validation callers can reject malformed input without changing execution parsing. +pub(crate) fn split_v8_arg_string(raw: &str) -> (Vec, bool) { + let mut args = Vec::new(); + let mut current = String::new(); + let mut in_quotes = false; + + for character in raw.chars() { + match character { + '"' => in_quotes = !in_quotes, + value if value.is_whitespace() && !in_quotes => { + if !current.is_empty() { + args.push(std::mem::take(&mut current)); + } + } + value => current.push(value), + } + } + + if !current.is_empty() { + args.push(current); + } + + (args, !in_quotes) +} diff --git a/src/support/error.rs b/src/support/error.rs index b67ffbf..4cb196b 100644 --- a/src/support/error.rs +++ b/src/support/error.rs @@ -1,4 +1,5 @@ use crate::config::loader::ConfigLoadError; +use crate::domain::runtime_state::RuntimeStateError; use crate::platform::designer::DesignerError; use crate::platform::edt::EdtError; use crate::platform::edt_session::EdtSessionError; @@ -157,6 +158,18 @@ impl AppError { } } +impl From for AppError { + fn from(error: RuntimeStateError) -> Self { + match error { + RuntimeStateError::PathResolution(_) => Self::Runtime(error.to_string()), + RuntimeStateError::EmptyConnection + | RuntimeStateError::MalformedConnectionString + | RuntimeStateError::MalformedRawConnection + | RuntimeStateError::UnsupportedRawConnection => Self::Validation(error.to_string()), + } + } +} + impl From for AppError { fn from(error: IbcmdError) -> Self { match error { diff --git a/src/support/mod.rs b/src/support/mod.rs index a1e6035..8492d1d 100644 --- a/src/support/mod.rs +++ b/src/support/mod.rs @@ -1,4 +1,5 @@ pub mod adapter_input; +pub mod connection_args; pub mod edt_project; pub mod error; pub mod fs; @@ -7,3 +8,5 @@ pub mod path; pub mod source_descriptor; pub mod temp; pub mod time; +#[cfg(windows)] +pub(crate) mod windows_fs; diff --git a/src/support/path.rs b/src/support/path.rs index 7467b5d..7633f85 100644 --- a/src/support/path.rs +++ b/src/support/path.rs @@ -31,43 +31,34 @@ pub fn nearest_existing_canonical_path(path: &Path) -> std::io::Result std::env::current_dir()?.join(path) }; - let mut existing = absolute.as_path(); - while !existing.exists() { - existing = existing.parent().ok_or_else(|| { - std::io::Error::new( - std::io::ErrorKind::NotFound, - format!("no existing ancestor for path '{}'", path.display()), - ) - })?; - } - - let existing_canonical = std::fs::canonicalize(existing)?; - if existing == absolute { - return Ok(existing_canonical); - } - - let suffix = absolute - .strip_prefix(existing) - .map_err(|error| std::io::Error::new(std::io::ErrorKind::InvalidInput, error))?; - let suffix = - suffix - .components() - .try_fold(PathBuf::new(), |mut acc, component| match component { - Component::Normal(part) => { - acc.push(part); - Ok(acc) + let mut resolved = PathBuf::new(); + for component in absolute.components() { + match component { + Component::Prefix(prefix) => resolved.push(prefix.as_os_str()), + Component::RootDir => resolved.push(component.as_os_str()), + Component::CurDir => {} + Component::ParentDir => { + if !resolved.pop() { + return Err(std::io::Error::new( + std::io::ErrorKind::InvalidInput, + format!("path escapes filesystem root: '{}'", path.display()), + )); + } + if resolved.try_exists()? { + resolved = std::fs::canonicalize(&resolved)?; } - _ => Err(std::io::Error::new( - std::io::ErrorKind::InvalidInput, - format!( - "path '{}' contains unsupported component '{}'", - path.display(), - component.as_os_str().to_string_lossy() - ), - )), - })?; - - Ok(existing_canonical.join(suffix)) + } + Component::Normal(part) => { + let candidate = resolved.join(part); + resolved = if candidate.try_exists()? { + std::fs::canonicalize(candidate)? + } else { + candidate + }; + } + } + } + Ok(resolved) } pub fn stable_path_identity(path: &Path) -> String { @@ -136,7 +127,18 @@ mod tests { let resolved = nearest_existing_canonical_path(&root.join("nested").join("target")).expect("resolved"); - assert_eq!(resolved, root.join("nested").join("target")); + let canonical_root = fs::canonicalize(&root).expect("canonical root"); + assert_eq!(resolved, canonical_root.join("nested").join("target")); + } + + #[cfg(unix)] + #[test] + fn nearest_existing_canonical_path_propagates_lookup_errors() { + let dir = tempdir().expect("tempdir"); + let path = dir.path().join("loop"); + std::os::unix::fs::symlink(&path, &path).expect("self symlink"); + + assert!(nearest_existing_canonical_path(&path).is_err()); } #[cfg(unix)] diff --git a/src/support/windows_fs.rs b/src/support/windows_fs.rs new file mode 100644 index 0000000..5135793 --- /dev/null +++ b/src/support/windows_fs.rs @@ -0,0 +1,387 @@ +use std::ffi::{OsStr, OsString}; +use std::fs::{File, OpenOptions}; +use std::io::{self, Read, Write}; +use std::mem::{offset_of, size_of}; +use std::os::windows::ffi::OsStrExt; +use std::os::windows::fs::{MetadataExt, OpenOptionsExt}; +use std::os::windows::io::{AsRawHandle, FromRawHandle}; +use std::path::{Component, Path}; +use std::ptr; + +use windows_sys::Wdk::Foundation::OBJECT_ATTRIBUTES; +use windows_sys::Wdk::Storage::FileSystem::{ + NtCreateFile, NtFlushBuffersFile, RtlNtStatusToDosErrorNoTeb, FILE_CREATE, FILE_DIRECTORY_FILE, + FILE_NON_DIRECTORY_FILE, FILE_OPEN, FILE_OPEN_IF, FILE_OPEN_REPARSE_POINT, + FILE_SYNCHRONOUS_IO_NONALERT, +}; +use windows_sys::Win32::Foundation::{ + HANDLE, OBJ_CASE_INSENSITIVE, STATUS_OBJECT_NAME_NOT_FOUND, UNICODE_STRING, +}; +use windows_sys::Win32::Storage::FileSystem::{ + FileDispositionInfoEx, FileRenameInfoEx, SetFileInformationByHandle, DELETE, + FILE_ATTRIBUTE_NORMAL, FILE_DISPOSITION_FLAG_DELETE, FILE_DISPOSITION_FLAG_POSIX_SEMANTICS, + FILE_DISPOSITION_INFO_EX, FILE_FLAG_BACKUP_SEMANTICS, FILE_FLAG_OPEN_REPARSE_POINT, + FILE_LIST_DIRECTORY, FILE_READ_ATTRIBUTES, FILE_READ_DATA, FILE_RENAME_INFO, + FILE_RENAME_INFO_0, FILE_SHARE_DELETE, FILE_SHARE_READ, FILE_SHARE_WRITE, FILE_TRAVERSE, + FILE_WRITE_ATTRIBUTES, FILE_WRITE_DATA, SYNCHRONIZE, +}; +use windows_sys::Win32::System::IO::{IO_STATUS_BLOCK, IO_STATUS_BLOCK_0}; + +const REPARSE_POINT_ATTRIBUTE: u32 = 0x0000_0400; + +pub(crate) struct ParentHandle { + pub(crate) directory: File, + pub(crate) file_name: OsString, +} + +pub(crate) fn open_root(root: &Path) -> io::Result { + let mut options = OpenOptions::new(); + options + .read(true) + .custom_flags(FILE_FLAG_BACKUP_SEMANTICS | FILE_FLAG_OPEN_REPARSE_POINT); + let file = options.open(root)?; + let metadata = file.metadata()?; + if !metadata.is_dir() || metadata.file_attributes() & REPARSE_POINT_ATTRIBUTE != 0 { + return Err(io::Error::new( + io::ErrorKind::InvalidData, + "managed root is not a non-reparse directory", + )); + } + Ok(file) +} + +pub(crate) fn open_parent( + root: &Path, + relative: &str, + create: bool, +) -> io::Result> { + let mut directory = open_root(root)?; + let components = Path::new(relative).components().collect::>(); + let Some((file_component, parents)) = components.split_last() else { + return Err(io::Error::new( + io::ErrorKind::InvalidInput, + "empty managed path", + )); + }; + for component in parents { + let Component::Normal(name) = component else { + return Err(io::Error::new( + io::ErrorKind::InvalidInput, + "non-normal managed path", + )); + }; + directory = match nt_open_relative( + &directory, + name, + FILE_LIST_DIRECTORY | FILE_TRAVERSE | FILE_READ_ATTRIBUTES | SYNCHRONIZE, + if create { FILE_OPEN_IF } else { FILE_OPEN }, + FILE_DIRECTORY_FILE | FILE_OPEN_REPARSE_POINT | FILE_SYNCHRONOUS_IO_NONALERT, + FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, + ) { + Ok(directory) => directory, + Err(error) if error.kind() == io::ErrorKind::NotFound && !create => return Ok(None), + Err(error) => return Err(error), + }; + directory = ensure_non_reparse_directory(directory)?; + } + let Component::Normal(file_name) = file_component else { + return Err(io::Error::new( + io::ErrorKind::InvalidInput, + "non-normal managed file", + )); + }; + Ok(Some(ParentHandle { + directory, + file_name: file_name.to_os_string(), + })) +} + +pub(crate) fn read_optional(parent: &ParentHandle) -> io::Result>> { + read_named_optional(&parent.directory, &parent.file_name) +} + +pub(crate) fn read_named_optional(parent: &File, name: &OsStr) -> io::Result>> { + let mut file = match open_regular(parent, name, FILE_OPEN, false) { + Ok(file) => file, + Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(None), + Err(error) => return Err(error), + }; + read_all(&mut file).map(Some) +} + +pub(crate) fn read_all(file: &mut File) -> io::Result> { + let mut bytes = Vec::new(); + file.read_to_end(&mut bytes)?; + Ok(bytes) +} + +pub(crate) fn create_regular(parent: &ParentHandle) -> io::Result { + create_named(&parent.directory, &parent.file_name) +} + +pub(crate) fn create_named(parent: &File, name: &OsStr) -> io::Result { + open_regular(parent, name, FILE_CREATE, true) +} + +pub(crate) fn open_regular_existing(parent: &ParentHandle, delete: bool) -> io::Result { + open_named_existing(&parent.directory, &parent.file_name, delete) +} + +pub(crate) fn open_regular_existing_for_metadata(parent: &ParentHandle) -> io::Result { + nt_open_relative( + &parent.directory, + &parent.file_name, + FILE_READ_DATA | FILE_READ_ATTRIBUTES | FILE_WRITE_ATTRIBUTES | SYNCHRONIZE, + FILE_OPEN, + FILE_NON_DIRECTORY_FILE | FILE_OPEN_REPARSE_POINT | FILE_SYNCHRONOUS_IO_NONALERT, + FILE_SHARE_READ | FILE_SHARE_DELETE, + ) + .and_then(ensure_non_reparse_file) +} + +pub(crate) fn open_named_existing(parent: &File, name: &OsStr, delete: bool) -> io::Result { + let access = FILE_READ_DATA + | FILE_READ_ATTRIBUTES + | SYNCHRONIZE + | if delete { + DELETE | FILE_WRITE_ATTRIBUTES + } else { + 0 + }; + nt_open_relative( + parent, + name, + access, + FILE_OPEN, + FILE_NON_DIRECTORY_FILE | FILE_OPEN_REPARSE_POINT | FILE_SYNCHRONOUS_IO_NONALERT, + FILE_SHARE_READ | FILE_SHARE_DELETE, + ) + .and_then(ensure_non_reparse_file) +} + +pub(crate) fn remove_directory(parent: &ParentHandle) -> io::Result<()> { + let directory = nt_open_relative( + &parent.directory, + &parent.file_name, + FILE_LIST_DIRECTORY | FILE_READ_ATTRIBUTES | DELETE | SYNCHRONIZE, + FILE_OPEN, + FILE_DIRECTORY_FILE | FILE_OPEN_REPARSE_POINT | FILE_SYNCHRONOUS_IO_NONALERT, + FILE_SHARE_READ | FILE_SHARE_DELETE, + ) + .and_then(ensure_non_reparse_directory)?; + delete_on_close(&directory)?; + flush(&parent.directory) +} + +pub(crate) fn rename_to(file: &File, parent: &File, name: &OsStr) -> io::Result<()> { + let encoded = encode_component(name)?; + let offset = offset_of!(FILE_RENAME_INFO, FileName); + let byte_size = offset + encoded.len() * size_of::(); + let word_count = byte_size.div_ceil(size_of::()); + // FILE_RENAME_INFO contains pointer-sized fields and must not be placed in a byte-aligned Vec. + let mut buffer = vec![0_usize; word_count]; + let info = buffer.as_mut_ptr().cast::(); + unsafe { + (*info).Anonymous = FILE_RENAME_INFO_0 { Flags: 0 }; + (*info).RootDirectory = parent.as_raw_handle() as HANDLE; + (*info).FileNameLength = (encoded.len() * size_of::()) as u32; + ptr::copy_nonoverlapping( + encoded.as_ptr(), + buffer.as_mut_ptr().cast::().add(offset).cast(), + encoded.len(), + ); + if SetFileInformationByHandle( + file.as_raw_handle() as HANDLE, + FileRenameInfoEx, + buffer.as_ptr().cast(), + byte_size as u32, + ) == 0 + { + return Err(io::Error::last_os_error()); + } + } + flush(parent) +} + +pub(crate) fn delete_on_close(file: &File) -> io::Result<()> { + let info = FILE_DISPOSITION_INFO_EX { + Flags: FILE_DISPOSITION_FLAG_DELETE | FILE_DISPOSITION_FLAG_POSIX_SEMANTICS, + }; + let result = unsafe { + SetFileInformationByHandle( + file.as_raw_handle() as HANDLE, + FileDispositionInfoEx, + (&info as *const FILE_DISPOSITION_INFO_EX).cast(), + size_of::() as u32, + ) + }; + if result == 0 { + Err(io::Error::last_os_error()) + } else { + Ok(()) + } +} + +pub(crate) fn same_directory(left: &File, right: &File) -> io::Result { + let left = left.metadata()?; + let right = right.metadata()?; + Ok(left.volume_serial_number() == right.volume_serial_number() + && left.file_index() == right.file_index()) +} + +fn open_regular(parent: &File, name: &OsStr, disposition: u32, write: bool) -> io::Result { + let access = FILE_READ_DATA + | FILE_READ_ATTRIBUTES + | SYNCHRONIZE + | if write { + FILE_WRITE_DATA | FILE_WRITE_ATTRIBUTES | DELETE + } else { + 0 + }; + nt_open_relative( + parent, + name, + access, + disposition, + FILE_NON_DIRECTORY_FILE | FILE_OPEN_REPARSE_POINT | FILE_SYNCHRONOUS_IO_NONALERT, + FILE_SHARE_READ | FILE_SHARE_DELETE, + ) + .and_then(ensure_non_reparse_file) +} + +fn nt_open_relative( + parent: &File, + name: &OsStr, + access: u32, + disposition: u32, + options: u32, + share: u32, +) -> io::Result { + let mut encoded = encode_component(name)?; + let mut unicode = UNICODE_STRING { + Length: (encoded.len() * size_of::()) as u16, + MaximumLength: (encoded.len() * size_of::()) as u16, + Buffer: encoded.as_mut_ptr(), + }; + let attributes = OBJECT_ATTRIBUTES { + Length: size_of::() as u32, + RootDirectory: parent.as_raw_handle() as HANDLE, + ObjectName: &mut unicode, + Attributes: OBJ_CASE_INSENSITIVE, + SecurityDescriptor: ptr::null(), + SecurityQualityOfService: ptr::null(), + }; + let mut handle: HANDLE = ptr::null_mut(); + let mut status_block = IO_STATUS_BLOCK { + Anonymous: IO_STATUS_BLOCK_0 { Status: 0 }, + Information: 0, + }; + let status = unsafe { + NtCreateFile( + &mut handle, + access, + &attributes, + &mut status_block, + ptr::null(), + FILE_ATTRIBUTE_NORMAL, + share, + disposition, + options, + ptr::null(), + 0, + ) + }; + if status < 0 { + let code = unsafe { RtlNtStatusToDosErrorNoTeb(status) }; + return Err(io::Error::from_raw_os_error(code as i32)); + } + if handle.is_null() || status == STATUS_OBJECT_NAME_NOT_FOUND { + return Err(io::Error::new( + io::ErrorKind::NotFound, + "managed entry not found", + )); + } + Ok(unsafe { File::from_raw_handle(handle.cast()) }) +} + +fn ensure_non_reparse_directory(file: File) -> io::Result { + let metadata = file.metadata()?; + if metadata.is_dir() && metadata.file_attributes() & REPARSE_POINT_ATTRIBUTE == 0 { + Ok(file) + } else { + Err(io::Error::new( + io::ErrorKind::InvalidData, + "managed parent is a reparse point", + )) + } +} + +fn ensure_non_reparse_file(file: File) -> io::Result { + let metadata = file.metadata()?; + if metadata.is_file() && metadata.file_attributes() & REPARSE_POINT_ATTRIBUTE == 0 { + Ok(file) + } else { + Err(io::Error::new( + io::ErrorKind::InvalidData, + "managed entry is not a regular file", + )) + } +} + +pub(crate) fn write_synced(mut file: File, bytes: &[u8]) -> io::Result { + file.write_all(bytes)?; + file.sync_all()?; + Ok(file) +} + +pub(crate) fn flush(file: &File) -> io::Result<()> { + let mut status_block = IO_STATUS_BLOCK { + Anonymous: IO_STATUS_BLOCK_0 { Status: 0 }, + Information: 0, + }; + let status = unsafe { NtFlushBuffersFile(file.as_raw_handle() as HANDLE, &mut status_block) }; + if status < 0 { + let code = unsafe { RtlNtStatusToDosErrorNoTeb(status) }; + Err(io::Error::from_raw_os_error(code as i32)) + } else { + Ok(()) + } +} + +fn encode_component(name: &OsStr) -> io::Result> { + let encoded = name.encode_wide().collect::>(); + if encoded.is_empty() + || encoded.contains(&0) + || encoded.iter().any(|unit| { + *unit == u16::from(b'/') || *unit == u16::from(b'\\') || *unit == u16::from(b':') + }) + || matches!(encoded.last(), Some(unit) if *unit == b'.' as u16 || *unit == b' ' as u16) + || encoded.len().saturating_mul(size_of::()) > u16::MAX as usize + || is_dos_device_name(&encoded) + { + return Err(io::Error::new( + io::ErrorKind::InvalidInput, + "invalid Windows managed path component", + )); + } + Ok(encoded) +} + +fn is_dos_device_name(encoded: &[u16]) -> bool { + let stem = encoded + .split(|unit| *unit == b'.' as u16) + .next() + .unwrap_or_default(); + let upper = stem + .iter() + .map(|unit| { + char::from_u32(u32::from(*unit)) + .unwrap_or('\0') + .to_ascii_uppercase() + }) + .collect::(); + matches!(upper.as_str(), "CON" | "PRN" | "AUX" | "NUL") + || (upper.len() == 4 + && (upper.starts_with("COM") || upper.starts_with("LPT")) + && matches!(upper.as_bytes()[3], b'1'..=b'9')) +} diff --git a/src/use_cases/artifacts.rs b/src/use_cases/artifacts.rs index db4cfff..a1336e8 100644 --- a/src/use_cases/artifacts.rs +++ b/src/use_cases/artifacts.rs @@ -640,7 +640,7 @@ fn resolve_target( args: &ArtifactsRequest, ) -> Result { let output_path = validate_output_path(args)?; - let inventory = SourceSetInventory::new(config); + let inventory = SourceSetInventory::new(config).map_err(AppError::from)?; let (source_set, extension) = match args.mode { ArtifactsModeRequest::ConfigurationCf => { diff --git a/src/use_cases/build_project.rs b/src/use_cases/build_project.rs index ead66b2..297932f 100644 --- a/src/use_cases/build_project.rs +++ b/src/use_cases/build_project.rs @@ -27,7 +27,14 @@ use crate::use_cases::external_artifacts::{ }; use crate::use_cases::request::BuildRequest as BuildArgs; use crate::use_cases::result::{UseCaseFailure, UseCaseResult}; +use crate::use_cases::runtime_state::{ + cleanup_orphan_designer_transactions, commit_designer_state_with_lock, + designer_full_rebuild_required, inspect_private_cdfi, lock_designer_state, + recover_designer_state, recover_designer_state_with_lock, require_designer_full_rebuild, + PrivateCdfiState, ValidatedCdfi, +}; use crate::use_cases::source_inventory::SourceSetInventory; +use crate::use_cases::source_transaction::{CdfiSeed, DesignerSourceTransaction}; use crate::use_cases::tool_extension; use tempfile::NamedTempFile; use tracing::debug; @@ -37,10 +44,12 @@ mod helpers; pub(crate) use self::helpers::ensure_platform_success; use self::helpers::{ - build_designer_dsl, build_ibcmd_dsl, commit_step_state, deferred_interruption_warning, - extension_name, fail_from_source_set_index, interruption_before_safe_point, map_ibcmd_error, - merge_step_message, plan_configurator_load_step, plan_edt_export_step, - plan_generated_designer_load_step, push_build_step, remove_storage_path, StepCommit, StepPlan, + attach_failed_plan_receipt, attach_failed_receipt, build_designer_dsl, build_ibcmd_dsl, + commit_step_state, deferred_interruption_warning, extension_name, fail_from_source_set_index, + interruption_before_safe_point, map_ibcmd_error, merge_step_message, + plan_configurator_load_step, plan_edt_export_step, plan_generated_designer_load_step, + push_build_step, push_build_step_with_receipt, receipt_after_success, remove_storage_path, + PlannedReceipt, StepCommit, StepPlan, }; #[cfg(test)] @@ -50,6 +59,57 @@ const SUPPORTED_DESIGNER_BUILD_ERROR: &str = const SUPPORTED_EDT_BUILD_ERROR: &str = "build with format=EDT currently supports only builder=DESIGNER or IBCMD"; +#[derive(Debug, Clone)] +enum DesignerCdfiPlan { + ForcedUnseeded, + Seeded(ValidatedCdfi), + IbcmdNormal, + IbcmdFull, +} + +impl DesignerCdfiPlan { + const fn forces_full_rebuild(&self) -> bool { + matches!(self, Self::ForcedUnseeded | Self::IbcmdFull) + } +} + +fn resolve_designer_cdfi_plan( + context: &SourceSetContext, + explicit_full_rebuild: bool, +) -> Result { + let state_lock = + lock_designer_state(context).map_err(|error| AppError::Runtime(error.to_string()))?; + recover_designer_state_with_lock(context, &state_lock) + .map_err(|error| AppError::Runtime(error.to_string()))?; + cleanup_orphan_designer_transactions(context, &state_lock) + .map_err(|error| AppError::Runtime(error.to_string()))?; + if explicit_full_rebuild { + return Ok(DesignerCdfiPlan::ForcedUnseeded); + } + if designer_full_rebuild_required(context) + .map_err(|error| AppError::Runtime(error.to_string()))? + { + debug!( + source_set = context.name(), + "previous Designer result was ambiguous; forcing an unseeded full load" + ); + return Ok(DesignerCdfiPlan::ForcedUnseeded); + } + match inspect_private_cdfi(&context.private_cdfi_path()) + .map_err(|error| AppError::Runtime(error.to_string()))? + { + PrivateCdfiState::Valid(cdfi) => Ok(DesignerCdfiPlan::Seeded(cdfi)), + PrivateCdfiState::Missing => Ok(DesignerCdfiPlan::ForcedUnseeded), + PrivateCdfiState::Corrupt(reason) => { + debug!( + source_set = context.name(), + reason, "private CDFI is invalid; forcing an unseeded full Designer load" + ); + Ok(DesignerCdfiPlan::ForcedUnseeded) + } + } +} + pub fn execute( context: &ExecutionContext, config: &AppConfig, @@ -428,6 +488,7 @@ fn execute_source_set_step( step_index: usize, partial_paths: Option<&[PathBuf]>, commit: &StepCommit, + cdfi_plan: &DesignerCdfiPlan, ) -> Result, AppError> { if let Some(error) = interruption_before_safe_point( context, @@ -452,6 +513,42 @@ fn execute_source_set_step( TimelineStageStatus::Running, ); } + let state_lock = lock_designer_state(commit_context) + .map_err(|error| AppError::Runtime(error.to_string()))?; + recover_designer_state_with_lock(commit_context, &state_lock) + .map_err(|error| AppError::Runtime(error.to_string()))?; + if let DesignerCdfiPlan::Seeded(expected) = cdfi_plan { + let current = inspect_private_cdfi(&commit_context.private_cdfi_path()) + .map_err(|error| AppError::Runtime(error.to_string()))?; + if !matches!(current, PrivateCdfiState::Valid(ref actual) if actual.bytes() == expected.bytes()) + { + return Err(AppError::Runtime(format!( + "private CDFI changed before Designer staging for source-set '{}'", + commit_context.name() + ))); + } + } + cleanup_orphan_designer_transactions(commit_context, &state_lock) + .map_err(|error| AppError::Runtime(error.to_string()))?; + let cdfi_seed = match cdfi_plan { + DesignerCdfiPlan::ForcedUnseeded => CdfiSeed::None, + DesignerCdfiPlan::Seeded(cdfi) => CdfiSeed::Validated(cdfi), + DesignerCdfiPlan::IbcmdNormal | DesignerCdfiPlan::IbcmdFull => { + return Err(AppError::Runtime( + "IBCMD CDFI plan cannot execute a Designer load".to_owned(), + )); + } + }; + let source_transaction = DesignerSourceTransaction::create( + load_context.path(), + load_context.excluded_roots(), + &commit_context.transactions_dir(), + cdfi_seed, + ) + .map_err(|error| AppError::Runtime(error.to_string()))?; + source_transaction + .verify_snapshot(commit.prepared_state()) + .map_err(|error| AppError::Runtime(error.to_string()))?; let load_result = if let Some(paths) = partial_paths { let list_file = partial_list_file(&config.work_path).map_err(|error| { AppError::Runtime(format!("failed to create partial list file: {error}")) @@ -473,8 +570,16 @@ fn execute_source_set_step( return Err(attach_partial_load_list_path(error, partial_list)); } }; + if let Some(error) = interruption_before_safe_point( + context, + format!("Designer load for source-set '{}'", source_set.name), + ) { + return Err(error); + } + require_designer_full_rebuild(commit_context) + .map_err(|error| AppError::Runtime(error.to_string()))?; let load_result = designer_dsl.load_config_from_files_partial( - load_context.path(), + source_transaction.load_root(), list_file.path(), extension_name(source_set), ); @@ -495,7 +600,7 @@ fn execute_source_set_step( } } } else { - build_designer_dsl( + let designer_dsl = build_designer_dsl( context, config, binary, @@ -504,9 +609,18 @@ fn execute_source_set_step( step_index, "load", InterruptionSafetyClass::CriticalNonAbortable, - )? - .load_config_from_files_full(load_context.path(), extension_name(source_set)) - .map_err(AppError::from)? + )?; + if let Some(error) = interruption_before_safe_point( + context, + format!("Designer load for source-set '{}'", source_set.name), + ) { + return Err(error); + } + require_designer_full_rebuild(commit_context) + .map_err(|error| AppError::Runtime(error.to_string()))?; + designer_dsl + .load_config_from_files_full(source_transaction.load_root(), extension_name(source_set)) + .map_err(AppError::from)? }; ensure_platform_success("load", source_set, &load_result)?; @@ -541,7 +655,17 @@ fn execute_source_set_step( .map_err(AppError::from)?; ensure_platform_success("update_db_cfg", source_set, &update_result)?; - commit_step_state(source_set, commit_context, &config.work_path, commit)?; + let prepared = commit.prepared_state(); + commit_designer_state_with_lock( + commit_context, + &state_lock, + prepared, + &source_transaction.load_root().join("ConfigDumpInfo.xml"), + ) + .map_err(|error| AppError::Runtime(error.to_string()))?; + source_transaction + .close() + .map_err(|error| AppError::Runtime(error.to_string()))?; Ok([ deferred_interruption_warning("load", &load_result), @@ -688,7 +812,7 @@ fn execute_source_set_step_ibcmd( .map_err(map_ibcmd_error)?; ensure_platform_success("apply", source_set, &apply_result)?; - commit_step_state(source_set, commit_context, &config.work_path, commit)?; + commit_step_state(source_set, commit_context, commit)?; Ok([ deferred_interruption_warning("ibcmd_import", &load_result), @@ -701,7 +825,11 @@ fn execute_source_set_step_ibcmd( #[cfg(test)] mod tests { + use super::helpers::{ + commit_step_state, plan_configurator_load_step, receipt_after_success, StepPlan, + }; use super::{run_build, BUILD_COMMAND}; + use crate::change_detection::analyzer::{self, AnalysisOutcome}; use crate::change_detection::hash_storage::{HashStorage, FILES_MTIME}; use crate::change_detection::source_sets::SourceSetsService; use crate::config::model::{ @@ -710,7 +838,6 @@ mod tests { ToolExtensionInput, ToolExtensionSourceConfig, ToolsConfig, }; use crate::domain::build::BuildMode; - use crate::domain::source_set::SourceSetContext; use crate::use_cases::context::{CommandName, ExecutionContext}; use crate::use_cases::request::BuildRequest as BuildArgs; use crate::use_cases::result::UseCaseErrorKind; @@ -742,7 +869,7 @@ mod tests { }) .unwrap_or_default(); let body = format!( - "args=\"$*\"\nout=\"\"\nprev=\"\"\nfor arg in \"$@\"; do\n if [ \"$prev\" = \"/Out\" ]; then out=\"$arg\"; fi\n prev=\"$arg\"\ndone\nif [ -n \"$out\" ]; then printf 'designer log for %s\\n' \"$args\" > \"$out\"; fi\nprintf '%s\\n' \"$args\" >> \"{}\"\n{}\nexit 0", + "args=\"$*\"\nout=\"\"\nload_root=\"\"\nprev=\"\"\nfor arg in \"$@\"; do\n if [ \"$prev\" = \"/Out\" ]; then out=\"$arg\"; fi\n if [ \"$prev\" = \"/LoadConfigFromFiles\" ]; then load_root=\"$arg\"; fi\n prev=\"$arg\"\ndone\nif [ -n \"$out\" ]; then printf 'designer log for %s\\n' \"$args\" > \"$out\"; fi\nseeded=no\nif [ -n \"$load_root\" ] && [ -f \"$load_root/ConfigDumpInfo.xml\" ]; then seeded=yes; fi\nprintf '%s seeded=%s\\n' \"$args\" \"$seeded\" >> \"{}\"\n{}\nif [ -n \"$load_root\" ]; then printf '%s\\n' '' > \"$load_root/ConfigDumpInfo.xml\"; fi\nexit 0", calls_log.display(), pattern_branch ); @@ -799,6 +926,20 @@ mod tests { make_executable(path); } + #[cfg(unix)] + fn write_delayed_edt_script(path: &Path, calls_log: &Path) { + let body = format!( + "args=\"$*\"\nproject=\"\"\ntarget=\"\"\nprev=\"\"\nfor arg in \"$@\"; do\n if [ \"$prev\" = \"--project-name\" ]; then project=\"$arg\"; fi\n if [ \"$prev\" = \"--configuration-files\" ]; then target=\"$arg\"; fi\n prev=\"$arg\"\ndone\nif [ -n \"$target\" ]; then mkdir -p \"$target\"; printf 'delayed export from %s\\n' \"$project\" > \"$target/exported.txt\"; printf '\\n' > \"$target/Configuration.xml\"; fi\nprintf '%s\\n' \"$args\" >> \"{}\"\nsleep 0.15\nexit 0", + calls_log.display() + ); + + if let Some(parent) = path.parent() { + fs::create_dir_all(parent).expect("create dirs"); + } + fs::write(path, format!("#!/bin/sh\n{body}\n")).expect("write delayed EDT script"); + make_executable(path); + } + #[cfg(unix)] fn write_edt_script_without_configuration(path: &Path, calls_log: &Path) { let body = format!( @@ -980,6 +1121,31 @@ mod tests { } } + #[test] + fn malformed_designer_connection_returns_structured_validation_failure_without_unwinding() { + let dir = tempdir().expect("tempdir"); + let mut config = build_config( + dir.path(), + &dir.path().join("work"), + &dir.path().join("1cv8"), + 20, + SourceFormat::Designer, + BuilderBackend::Designer, + ); + config.infobase.connection = "/Unsupported secret-value".to_owned(); + + let result = std::panic::catch_unwind(|| run_build(&config, &build_args(false))); + + assert!(result.is_ok(), "build boundary must not unwind"); + let failure = result + .expect("no unwind") + .expect_err("malformed connection must fail"); + assert_eq!(failure.error.kind(), UseCaseErrorKind::Validation); + let payload = failure.payload.expect("structured build payload"); + assert!(!payload.ok); + assert!(payload.steps.is_empty()); + } + #[cfg(unix)] #[test] fn execute_build_honors_interruption_before_load_safe_point() { @@ -1015,6 +1181,16 @@ mod tests { .error .message() .contains("before entering build load for source-set 'main' safe point")); + let payload = failure.payload.expect("cancelled build payload"); + let receipt = serde_json::to_value(&payload.steps[0].receipt).expect("receipt"); + assert_eq!(receipt["status"], "failed"); + assert!(!receipt["requested"] + .as_array() + .expect("requested") + .is_empty()); + assert_eq!(receipt["processed"], serde_json::json!([])); + assert_eq!(receipt["skipped"], serde_json::json!([])); + assert_eq!(receipt["conflicted"], serde_json::json!([])); assert!( !calls_log.exists() || fs::read_to_string(&calls_log) @@ -1077,6 +1253,14 @@ mod tests { .error .message() .contains("before entering ibcmd apply for source-set 'main' safe point")); + let payload = failure.payload.expect("cancelled ibcmd payload"); + let receipt = serde_json::to_value(&payload.steps[0].receipt).expect("receipt"); + assert_eq!(receipt["status"], "failed"); + assert!(!receipt["requested"] + .as_array() + .expect("requested") + .is_empty()); + assert_eq!(receipt["processed"], serde_json::json!([])); let calls = fs::read_to_string(&calls_log).expect("calls"); assert!(calls.contains("config import")); assert!(!calls.contains("config apply")); @@ -1265,6 +1449,16 @@ mod tests { let export_root = &export_roots[0]; assert!(result.ok); + let receipt = serde_json::to_value(&result.steps[0].receipt).expect("receipt"); + assert_eq!(receipt["status"], "applied"); + assert!(!receipt["requested"] + .as_array() + .expect("requested") + .is_empty()); + assert!(!receipt["processed"] + .as_array() + .expect("processed") + .is_empty()); assert!(result.steps.iter().any(|step| { step.source_set == "processors" && matches!(step.mode, BuildMode::EdtExport) && step.ok })); @@ -1273,29 +1467,38 @@ mod tests { } fn prime_snapshots(config: &AppConfig) { - let service = SourceSetsService::new(config); - for context in service.designer_contexts() { - crate::change_detection::analyzer::rescan_and_commit_full(&context, &config.work_path) + let service = SourceSetsService::new(config).expect("service"); + for context in service.designer_contexts().expect("contexts") { + crate::change_detection::analyzer::rescan_and_commit_full(&context) .expect("prime snapshot"); + if let Some(parent) = context.private_cdfi_path().parent() { + fs::create_dir_all(parent).expect("private state dir"); + } + fs::write( + context.private_cdfi_path(), + r#""#, + ) + .expect("prime private CDFI"); } } fn prime_edt_snapshots(config: &AppConfig) { - let service = SourceSetsService::new(config); - for context in service.edt_contexts() { - crate::change_detection::analyzer::rescan_and_commit_full(&context, &config.work_path) + let service = SourceSetsService::new(config).expect("service"); + for context in service.edt_contexts().expect("contexts") { + crate::change_detection::analyzer::rescan_and_commit_full(&context) .expect("prime edt snapshot"); } } fn storage_generation(config: &AppConfig, source_set_name: &str) -> u64 { - let service = SourceSetsService::new(config); + let service = SourceSetsService::new(config).expect("service"); let context = service .designer_contexts() + .expect("contexts") .into_iter() .find(|context| context.name() == source_set_name) .expect("context"); - HashStorage::new(context.storage_path(&config.work_path)) + HashStorage::new(context.storage_path()) .load_snapshot() .expect("snapshot") .generation @@ -1318,16 +1521,25 @@ mod tests { source_path: &Path, extension_name: &str, ) -> PathBuf { - let context = SourceSetContext::new( - format!("tool:{extension_name}"), - source_path.to_path_buf(), - format!("tool-{extension_name}-source"), - ); - context.storage_path(&config.work_path) + let extension = config + .tools + .client_mcp + .extension + .as_ref() + .filter(|extension| extension.name == extension_name) + .expect("configured tool extension"); + let source = extension.source().expect("source tool extension"); + let context = crate::use_cases::tool_extension::tool_extension_source_context( + config, extension, source, + ) + .expect("tool extension context"); + assert_eq!(context.path(), source_path); + context.storage_path() } fn write_recoverable_tool_extension_storage(path: &Path) { remove_file_if_exists(path); + std::fs::create_dir_all(path.parent().expect("storage parent")).expect("storage parent"); let db = redb::Database::create(path).expect("create recoverable storage"); let tx = db.begin_write().expect("begin recoverable storage write"); { @@ -1338,18 +1550,224 @@ mod tests { } fn edt_storage_generation(config: &AppConfig, source_set_name: &str) -> u64 { - let service = SourceSetsService::new(config); + let service = SourceSetsService::new(config).expect("service"); let context = service .edt_contexts() + .expect("contexts") .into_iter() .find(|context| context.name() == source_set_name) .expect("edt context"); - HashStorage::new(context.storage_path(&config.work_path)) + HashStorage::new(context.storage_path()) .load_snapshot() .expect("snapshot") .generation } + fn runtime_storage_paths(config: &AppConfig) -> Vec { + let service = SourceSetsService::new(config).expect("service"); + let mut paths = service + .designer_contexts() + .expect("designer contexts") + .into_iter() + .chain(service.edt_contexts().expect("EDT contexts")) + .map(|context| context.storage_path()) + .collect::>(); + paths.sort(); + paths + } + + fn storage_generations(paths: &[PathBuf]) -> Vec { + paths + .iter() + .map(|path| { + HashStorage::new(path.clone()) + .current_generation() + .expect("runtime-state generation") + }) + .collect() + } + + fn assert_source_set_skipped(result: &crate::domain::build::BuildResult) { + let source_steps = result + .steps + .iter() + .filter(|step| step.source_set == "main") + .collect::>(); + assert!(!source_steps.is_empty(), "main source-set must have steps"); + assert!(source_steps + .iter() + .all(|step| step.ok && matches!(step.mode, BuildMode::Skipped))); + } + + #[cfg(unix)] + #[test] + fn per_infobase_lifecycle_isolated_for_all_format_backend_pairs() { + for (format, backend) in [ + (SourceFormat::Designer, BuilderBackend::Designer), + (SourceFormat::Designer, BuilderBackend::Ibcmd), + (SourceFormat::Edt, BuilderBackend::Designer), + (SourceFormat::Edt, BuilderBackend::Ibcmd), + ] { + let dir = tempdir().expect("tempdir"); + let base = dir.path().join("base"); + let work = dir.path().join("work"); + let designer = dir.path().join("platform").join("bin").join("1cv8"); + let ibcmd = dir.path().join("ibcmd"); + let edt = dir.path().join("edt").join("1cedtcli"); + let platform_calls = dir.path().join("platform-calls.log"); + let edt_calls = dir.path().join("edt-calls.log"); + create_source_tree(&base); + write_designer_script(&designer, &platform_calls, None); + write_ibcmd_script(&ibcmd, &platform_calls, None); + write_edt_script(&edt, &edt_calls, None); + let platform_path = match backend { + BuilderBackend::Designer => dir.path().join("platform"), + BuilderBackend::Ibcmd => ibcmd.clone(), + }; + let mut config_a = match format { + SourceFormat::Designer => { + build_config(&base, &work, &platform_path, 20, format, backend) + } + SourceFormat::Edt => { + let mut config = build_edt_config(&base, &work, &platform_path, &edt); + config.builder = backend; + config + } + }; + config_a.source_sets.truncate(1); + config_a.infobase = crate::config::model::InfobaseConfig::file(format!( + "File={}", + dir.path().join("ib-a").display() + )); + let mut config_b = config_a.clone(); + config_b.infobase = crate::config::model::InfobaseConfig::file(format!( + "File={}", + dir.path().join("ib-b").display() + )); + + let first_a = run_build(&config_a, &build_args(false)).expect("A bootstrap build"); + assert!(first_a.ok); + assert!(first_a.steps.iter().any(|step| { + step.source_set == "main" && step.ok && matches!(step.mode, BuildMode::Full) + })); + assert_source_set_skipped( + &run_build(&config_a, &build_args(false)).expect("A repeat skip"), + ); + let a_paths = runtime_storage_paths(&config_a); + let a_generations = storage_generations(&a_paths); + + let first_b = run_build(&config_b, &build_args(false)).expect("B bootstrap build"); + assert!(first_b.steps.iter().any(|step| { + step.source_set == "main" && step.ok && matches!(step.mode, BuildMode::Full) + })); + let b_paths = runtime_storage_paths(&config_b); + assert!(a_paths.iter().all(|path| !b_paths.contains(path))); + assert_eq!(storage_generations(&a_paths), a_generations); + + assert_source_set_skipped( + &run_build(&config_a, &build_args(false)).expect("A remains skipped after B"), + ); + let restarted_a = config_a.clone(); + drop(config_a); + assert_source_set_skipped( + &run_build(&restarted_a, &build_args(false)).expect("A restart skip"), + ); + assert_eq!(storage_generations(&a_paths), a_generations); + } + } + + #[cfg(unix)] + #[test] + fn legacy_hash_storages_are_never_read_or_mutated() { + let dir = tempdir().expect("tempdir"); + let base = dir.path().join("base"); + let work = dir.path().join("work"); + let platform = dir.path().join("platform").join("bin").join("1cv8"); + let edt = dir.path().join("edt").join("1cedtcli"); + let designer_calls = dir.path().join("designer-calls.log"); + let edt_calls = dir.path().join("edt-calls.log"); + let tool_source = base.join("tool-client-mcp"); + create_source_tree(&base); + create_edt_tool_extension_source(&tool_source); + write_designer_script(&platform, &designer_calls, None); + write_edt_script(&edt, &edt_calls, None); + let mut config = build_edt_config(&base, &work, &dir.path().join("platform"), &edt); + config.source_sets.truncate(1); + config.tools.client_mcp.extension = Some(ToolExtensionConfig { + name: "client_mcp".to_owned(), + input: ToolExtensionInput::Source(ToolExtensionSourceConfig { + path: PathBuf::from("tool-client-mcp"), + format: Some(SourceFormat::Edt), + }), + }); + let legacy_root = work.join("hash-storages"); + fs::create_dir_all(&legacy_root).expect("legacy root"); + let sentinels = [ + ("designer-main.redb", b"legacy designer sentinel".as_slice()), + ("edt-main.redb", b"legacy EDT sentinel".as_slice()), + ( + "tool-client_mcp-source.redb", + b"legacy tool sentinel".as_slice(), + ), + ]; + for (name, bytes) in sentinels { + fs::write(legacy_root.join(name), bytes).expect("legacy sentinel"); + } + + let first = run_build(&config, &build_args(false)).expect("new-layout bootstrap"); + assert!(first.steps.iter().any(|step| { + step.source_set == "main" && step.ok && matches!(step.mode, BuildMode::Full) + })); + let repeat = run_build(&config, &build_args(false)).expect("new-layout repeat"); + assert_source_set_skipped(&repeat); + assert!(repeat.steps.iter().any(|step| { + step.source_set == "tool:client_mcp" + && step.ok + && matches!(step.mode, BuildMode::Skipped) + })); + let tool_a_path = tool_extension_storage_path(&config, &tool_source, "client_mcp"); + let tool_a_generation = HashStorage::new(tool_a_path.clone()) + .current_generation() + .expect("A tool generation"); + let mut config_b = config.clone(); + config_b.infobase = crate::config::model::InfobaseConfig::file(format!( + "File={}", + dir.path().join("legacy-test-ib-b").display() + )); + let first_b = run_build(&config_b, &build_args(false)).expect("B new-layout bootstrap"); + assert!(first_b.steps.iter().any(|step| { + step.source_set == "tool:client_mcp" + && step.ok + && !matches!(step.mode, BuildMode::Skipped) + })); + let tool_b_path = tool_extension_storage_path(&config_b, &tool_source, "client_mcp"); + assert_ne!(tool_a_path, tool_b_path); + assert_eq!( + HashStorage::new(tool_a_path.clone()) + .current_generation() + .expect("unchanged A tool generation"), + tool_a_generation + ); + let after_b = run_build(&config, &build_args(false)).expect("A tool remains skipped"); + assert!(after_b.steps.iter().any(|step| { + step.source_set == "tool:client_mcp" + && step.ok + && matches!(step.mode, BuildMode::Skipped) + })); + + for (name, bytes) in sentinels { + assert_eq!( + fs::read(legacy_root.join(name)).expect("unchanged sentinel"), + bytes + ); + } + let mut new_paths = runtime_storage_paths(&config); + new_paths.push(tool_a_path); + new_paths.push(tool_b_path); + assert!(new_paths.iter().all(|path| path.exists())); + assert!(new_paths.iter().all(|path| !path.starts_with(&legacy_root))); + } + #[cfg(unix)] #[test] fn ibcmd_build_dispatch_uses_ibcmd_utility() { @@ -1371,6 +1789,16 @@ mod tests { let result = run_build(&config, &build_args(true)).expect("build"); assert!(result.ok); + let receipt = serde_json::to_value(&result.steps[0].receipt).expect("receipt"); + assert_eq!(receipt["status"], "applied"); + assert!(!receipt["requested"] + .as_array() + .expect("requested") + .is_empty()); + assert!(!receipt["processed"] + .as_array() + .expect("processed") + .is_empty()); let calls_text = fs::read_to_string(&calls).expect("calls"); assert!(calls_text.contains("config import")); assert!(calls_text.contains("config apply")); @@ -1467,7 +1895,7 @@ mod tests { config.tools.client_mcp.extension = Some(ToolExtensionConfig { name: "client_mcp".to_owned(), input: ToolExtensionInput::Source(ToolExtensionSourceConfig { - path: tool_source, + path: PathBuf::from("tool-client-mcp"), format: Some(SourceFormat::Edt), }), }); @@ -1479,7 +1907,8 @@ mod tests { let designer_calls_text = fs::read_to_string(&designer_calls).expect("designer calls"); assert!(edt_calls_text.contains("--project-name client-mcp-project")); assert!(edt_calls_text.contains("tool-extensions/client_mcp")); - assert!(designer_calls_text.contains("tool-extensions/client_mcp")); + assert!(designer_calls_text.contains("/ib-state/v1/")); + assert!(designer_calls_text.contains("/transactions/designer-build-")); assert!(designer_calls_text.contains("-Extension client_mcp")); } @@ -1593,9 +2022,22 @@ mod tests { assert!(second.steps.iter().any(|step| { step.source_set == "tool:client_mcp" && matches!(step.mode, BuildMode::Full) && step.ok })); + let tool_step = second + .steps + .iter() + .find(|step| step.source_set == "tool:client_mcp") + .expect("tool step"); + let receipt = serde_json::to_value(&tool_step.receipt).expect("receipt"); + assert_eq!(receipt["status"], "applied"); + assert_eq!(receipt["requested"].as_array().expect("requested").len(), 1); + assert!(!receipt["processed"] + .as_array() + .expect("processed") + .is_empty()); assert!(edt_calls_text.contains("--project-name client-mcp-project")); assert!(edt_calls_text.contains("tool-extensions/client_mcp")); - assert!(designer_calls_text.contains("tool-extensions/client_mcp")); + assert!(designer_calls_text.contains("/ib-state/v1/")); + assert!(designer_calls_text.contains("/transactions/designer-build-")); assert!(designer_calls_text.contains("-Extension client_mcp")); assert_eq!( tool_extension_storage_generation(&config, &tool_source, "client_mcp"), @@ -1642,7 +2084,8 @@ mod tests { assert!(rebuild.ok); assert!(edt_calls_text.contains("--project-name client-mcp-project")); assert!(edt_calls_text.contains("tool-extensions/client_mcp")); - assert!(designer_calls_text.contains("tool-extensions/client_mcp")); + assert!(designer_calls_text.contains("/ib-state/v1/")); + assert!(designer_calls_text.contains("/transactions/designer-build-")); assert!(designer_calls_text.contains("-Extension client_mcp")); assert_eq!( tool_extension_storage_generation(&config, &tool_source, "client_mcp"), @@ -1701,6 +2144,16 @@ mod tests { let failure = run_build(&config, &build_args(false)).expect_err("failed export"); assert!(failure.error.message().contains("tool extension")); + let payload = failure.payload.expect("tool failure payload"); + let tool_step = payload + .steps + .iter() + .find(|step| step.source_set == "tool:client_mcp") + .expect("tool step"); + let receipt = serde_json::to_value(&tool_step.receipt).expect("receipt"); + assert_eq!(receipt["status"], "failed"); + assert_eq!(receipt["requested"].as_array().expect("requested").len(), 1); + assert_eq!(receipt["processed"], serde_json::json!([])); assert_eq!( tool_extension_storage_generation(&config, &tool_source, "client_mcp"), 1 @@ -1962,13 +2415,27 @@ mod tests { .steps .iter() .any(|step| matches!(step.mode, BuildMode::EdtExport) && step.ok)); + let export_step = result + .steps + .iter() + .find(|step| matches!(step.mode, BuildMode::EdtExport)) + .expect("export step"); + let receipt = serde_json::to_value(&export_step.receipt).expect("receipt"); + assert_eq!(receipt["status"], "applied"); + assert_eq!(receipt["requested"].as_array().expect("requested").len(), 1); + assert!(!receipt["processed"] + .as_array() + .expect("processed") + .is_empty()); assert!(result.steps.iter().any(|step| { step.source_set == "main" && matches!(step.mode, BuildMode::Full) && step.ok })); assert!(edt_calls_text.contains("export --project-name main")); assert!(designer_calls_text.contains("/LoadConfigFromFiles")); assert!(!designer_calls_text.contains("-partial")); - assert!(designer_calls_text.contains( + assert!(designer_calls_text.contains("/ib-state/v1/")); + assert!(designer_calls_text.contains("/transactions/designer-build-")); + assert!(!designer_calls_text.contains( work.join("designer") .join("main") .display() @@ -2315,25 +2782,59 @@ mod tests { .payload .expect("build failures should preserve a structured payload"); let designer_storage_path = SourceSetsService::new(&config) + .expect("service") .designer_contexts() + .expect("contexts") .into_iter() .find(|context| context.name() == "client_mcp") .expect("designer context") - .storage_path(&config.work_path); + .storage_path(); let designer_calls_text = fs::read_to_string(&designer_calls).expect("designer calls"); assert!(!result.ok); assert!(matches!(result.steps[0].mode, BuildMode::EdtExport)); assert!(result.steps[0].ok); - assert!(result.steps.iter().any(|step| { - step.source_set == "client_mcp" && matches!(step.mode, BuildMode::Full) && !step.ok - })); + let export_receipt = + serde_json::to_value(&result.steps[0].receipt).expect("export receipt"); + assert_eq!(export_receipt["status"], "applied"); + assert_eq!( + export_receipt["requested"] + .as_array() + .expect("requested") + .len(), + 1 + ); + let failed_step = result + .steps + .iter() + .find(|step| { + step.source_set == "client_mcp" && matches!(step.mode, BuildMode::Full) && !step.ok + }) + .expect("failed downstream step"); + let failed_receipt = serde_json::to_value(&failed_step.receipt).expect("failed receipt"); + assert_eq!(failed_receipt["status"], "failed"); + assert!(!failed_receipt["requested"] + .as_array() + .expect("requested") + .is_empty()); + assert_eq!(failed_receipt["processed"], serde_json::json!([])); assert!(!result.steps.iter().any(|step| { step.source_set == "client_mcp" && matches!(step.mode, BuildMode::Partial { .. }) })); assert!(designer_calls_text.contains("/UpdateDBCfg -Extension client_mcp")); - assert_eq!(edt_storage_generation(&config, "client_mcp"), 2); + assert_eq!(edt_storage_generation(&config, "client_mcp"), 1); assert!(!designer_storage_path.exists()); + + write_designer_script(&platform_script, &designer_calls, None); + run_build(&config, &build_args(false)).expect("extension retry succeeds"); + assert_eq!(edt_storage_generation(&config, "client_mcp"), 2); + assert_eq!( + fs::read_to_string(&edt_calls) + .expect("EDT calls") + .matches("export --project-name client_mcp") + .count(), + 2 + ); } #[cfg(unix)] @@ -2438,6 +2939,45 @@ mod tests { ); } + #[cfg(unix)] + #[test] + fn edt_snapshot_commits_when_generated_designer_output_is_already_converged() { + let dir = tempdir().expect("tempdir"); + let base = dir.path().join("base"); + let work = dir.path().join("work"); + let platform_script = dir.path().join("platform").join("bin").join("1cv8"); + let edt_script = dir.path().join("edt").join("1cedtcli"); + let designer_calls = dir.path().join("designer-calls.log"); + let edt_calls = dir.path().join("edt-calls.log"); + create_source_tree(&base); + write_designer_script(&platform_script, &designer_calls, None); + write_edt_script(&edt_script, &edt_calls, None); + let mut config = build_edt_config(&base, &work, &dir.path().join("platform"), &edt_script); + config.source_sets.truncate(1); + run_build(&config, &build_args(false)).expect("initial build"); + let generation_before = edt_storage_generation(&config, "main"); + fs::write( + base.join("main") + .join("Catalogs.Items") + .join("ObjectModule.bsl"), + "procedure Test()\n // EDT-only change with stable export\nendprocedure", + ) + .expect("modify EDT source"); + + let result = run_build(&config, &build_args(false)).expect("converged build"); + + assert!(result.steps.iter().any(|step| { + step.source_set == "main" && step.ok && matches!(step.mode, BuildMode::EdtExport) + })); + assert!(result.steps.iter().any(|step| { + step.source_set == "main" && step.ok && matches!(step.mode, BuildMode::Skipped) + })); + assert_eq!( + edt_storage_generation(&config, "main"), + generation_before + 1 + ); + } + #[cfg(unix)] #[test] fn edt_export_failure_stops_pipeline_before_designer_load() { @@ -2470,6 +3010,10 @@ mod tests { assert!(!result.ok); assert!(matches!(result.steps[0].mode, BuildMode::EdtExport)); assert!(!result.steps[0].ok); + let receipt = serde_json::to_value(&result.steps[0].receipt).expect("receipt"); + assert_eq!(receipt["status"], "failed"); + assert_eq!(receipt["requested"].as_array().expect("requested").len(), 1); + assert_eq!(receipt["processed"], serde_json::json!([])); assert!(!designer_calls.exists()); } @@ -2555,11 +3099,13 @@ mod tests { .payload .expect("build failures should preserve a structured payload"); let designer_storage_path = SourceSetsService::new(&config) + .expect("service") .designer_contexts() + .expect("contexts") .into_iter() .find(|context| context.name() == "main") .expect("designer context") - .storage_path(&config.work_path); + .storage_path(); assert!(!result.ok); assert!(matches!(result.steps[0].mode, BuildMode::EdtExport)); @@ -2567,8 +3113,256 @@ mod tests { assert!(result.steps.iter().any(|step| { step.source_set == "main" && matches!(step.mode, BuildMode::Full) && !step.ok })); - assert_eq!(edt_storage_generation(&config, "main"), 2); + assert_eq!( + serde_json::to_value(&result.steps[0].receipt).expect("EDT receipt")["status"], + "applied" + ); + let downstream = result + .steps + .iter() + .find(|step| step.source_set == "main" && !step.ok) + .expect("failed downstream step"); + let downstream_receipt = + serde_json::to_value(&downstream.receipt).expect("downstream receipt"); + assert_eq!(downstream_receipt["status"], "failed"); + assert_eq!(downstream_receipt["processed"], serde_json::json!([])); + assert_eq!(edt_storage_generation(&config, "main"), 1); assert!(!designer_storage_path.exists()); + + write_designer_script(&platform_script, &designer_calls, None); + run_build(&config, &build_args(false)).expect("retry succeeds"); + assert_eq!(edt_storage_generation(&config, "main"), 2); + assert_eq!( + fs::read_to_string(&edt_calls) + .expect("EDT calls") + .matches("export --project-name main") + .count(), + 2 + ); + } + + #[cfg(unix)] + #[test] + fn edt_ibcmd_apply_failure_keeps_edt_snapshot_pending_for_retry() { + let dir = tempdir().expect("tempdir"); + let base = dir.path().join("base"); + let work = dir.path().join("work"); + let ibcmd_script = dir.path().join("ibcmd"); + let edt_script = dir.path().join("edt").join("1cedtcli"); + let ibcmd_calls = dir.path().join("ibcmd-calls.log"); + let edt_calls = dir.path().join("edt-calls.log"); + create_source_tree(&base); + write_ibcmd_script(&ibcmd_script, &ibcmd_calls, Some("config apply")); + write_edt_script(&edt_script, &edt_calls, None); + let mut config = build_edt_config(&base, &work, &ibcmd_script, &edt_script); + config.builder = BuilderBackend::Ibcmd; + prime_edt_snapshots(&config); + fs::write( + base.join("main") + .join("Catalogs.Items") + .join("ObjectModule.bsl"), + "procedure Test()\n // changed in edt\nendprocedure", + ) + .expect("modify edt main"); + + let failure = run_build(&config, &build_args(false)).expect_err("apply must fail"); + let result = failure.payload.expect("structured failure"); + + assert_eq!(edt_storage_generation(&config, "main"), 1); + assert_eq!( + serde_json::to_value(&result.steps[0].receipt).expect("EDT receipt")["status"], + "applied" + ); + let downstream = result + .steps + .iter() + .find(|step| step.source_set == "main" && !step.ok) + .expect("failed IBCMD step"); + let downstream_receipt = serde_json::to_value(&downstream.receipt).expect("IBCMD receipt"); + assert_eq!(downstream_receipt["status"], "failed"); + assert_eq!(downstream_receipt["processed"], serde_json::json!([])); + write_ibcmd_script(&ibcmd_script, &ibcmd_calls, None); + run_build(&config, &build_args(false)).expect("retry succeeds"); + assert_eq!(edt_storage_generation(&config, "main"), 2); + assert_eq!( + fs::read_to_string(&edt_calls) + .expect("EDT calls") + .matches("export --project-name main") + .count(), + 2 + ); + } + + #[cfg(unix)] + #[test] + fn cancellation_after_edt_export_keeps_snapshot_pending_for_retry() { + let dir = tempdir().expect("tempdir"); + let base = dir.path().join("base"); + let work = dir.path().join("work"); + let ibcmd_script = dir.path().join("ibcmd"); + let edt_script = dir.path().join("edt").join("1cedtcli"); + let ibcmd_calls = dir.path().join("ibcmd-calls.log"); + let edt_calls = dir.path().join("edt-calls.log"); + create_source_tree(&base); + if let Some(parent) = ibcmd_script.parent() { + fs::create_dir_all(parent).expect("create ibcmd dir"); + } + fs::write( + &ibcmd_script, + format!( + "#!/bin/sh\nargs=\"$*\"\nprintf '%s\\n' \"$args\" >> \"{}\"\n\ + if printf '%s' \"$args\" | grep -F -q -- 'config import'; then sleep 0.15; fi\n\ + exit 0\n", + ibcmd_calls.display() + ), + ) + .expect("write delayed ibcmd script"); + make_executable(&ibcmd_script); + write_edt_script(&edt_script, &edt_calls, None); + let mut config = build_edt_config(&base, &work, &ibcmd_script, &edt_script); + config.builder = BuilderBackend::Ibcmd; + config.source_sets.truncate(1); + prime_edt_snapshots(&config); + fs::write( + base.join("main") + .join("Catalogs.Items") + .join("ObjectModule.bsl"), + "procedure Test()\n // cancel after export\nendprocedure", + ) + .expect("modify EDT source"); + let cancellation = CancellationToken::new(); + let delayed_cancel = cancellation.clone(); + let import_marker = ibcmd_calls.clone(); + let cancel_thread = thread::spawn(move || { + for _ in 0..4_000 { + if import_marker.exists() + && fs::read_to_string(&import_marker) + .expect("IBCMD calls") + .contains("config import") + { + delayed_cancel.cancel(); + return; + } + thread::sleep(Duration::from_millis(5)); + } + panic!("IBCMD import marker was not observed"); + }); + + let failure = super::execute( + &ExecutionContext::cli(CommandName::Build).with_cancellation(cancellation), + &config, + &build_args(false), + ) + .expect_err("build must stop before IBCMD apply"); + cancel_thread.join().expect("cancel thread"); + let result = failure.payload.expect("structured cancellation failure"); + + assert!(failure + .error + .message() + .contains("before entering ibcmd apply for source-set 'main' safe point")); + assert_eq!(edt_storage_generation(&config, "main"), 1); + assert_eq!( + serde_json::to_value(&result.steps[0].receipt).expect("EDT receipt")["status"], + "applied" + ); + let downstream = result.steps.last().expect("failed IBCMD step"); + let downstream_receipt = serde_json::to_value(&downstream.receipt).expect("IBCMD receipt"); + assert_eq!(downstream_receipt["status"], "failed"); + assert_eq!(downstream_receipt["processed"], serde_json::json!([])); + + run_build(&config, &build_args(false)).expect("retry succeeds"); + assert_eq!(edt_storage_generation(&config, "main"), 2); + assert_eq!( + fs::read_to_string(&edt_calls) + .expect("EDT calls") + .matches("export --project-name main") + .count(), + 2 + ); + } + + #[cfg(unix)] + #[test] + fn deferred_edt_commit_conflicts_use_edt_receipt_after_execute_and_skip() { + for generated_output_is_stable in [false, true] { + let dir = tempdir().expect("tempdir"); + let base = dir.path().join("base"); + let work = dir.path().join("work"); + let platform_script = dir.path().join("platform").join("bin").join("1cv8"); + let edt_script = dir.path().join("edt").join("1cedtcli"); + let designer_calls = dir.path().join("designer-calls.log"); + let edt_calls = dir.path().join("edt-calls.log"); + create_source_tree(&base); + write_designer_script(&platform_script, &designer_calls, None); + if generated_output_is_stable { + write_delayed_edt_script(&edt_script, &edt_calls); + } else { + write_edt_script(&edt_script, &edt_calls, None); + } + let mut config = + build_edt_config(&base, &work, &dir.path().join("platform"), &edt_script); + config.source_sets.truncate(1); + run_build(&config, &build_args(false)).expect("initial build"); + + fs::remove_file(&edt_calls).expect("clear EDT calls"); + if designer_calls.exists() { + fs::remove_file(&designer_calls).expect("clear Designer calls"); + } + write_delayed_edt_script(&edt_script, &edt_calls); + fs::write( + base.join("main") + .join("Catalogs.Items") + .join("ObjectModule.bsl"), + "procedure Test()\n // concurrent EDT commit\nendprocedure", + ) + .expect("modify EDT source"); + let edt_context = SourceSetsService::new(&config) + .expect("service") + .edt_contexts() + .expect("contexts") + .into_iter() + .find(|context| context.name() == "main") + .expect("EDT context"); + let concurrent_context = edt_context.clone(); + let calls_marker = edt_calls.clone(); + let concurrent_commit = thread::spawn(move || { + for _ in 0..4_000 { + if calls_marker.exists() { + analyzer::rescan_and_commit_full(&concurrent_context) + .expect("concurrent EDT commit"); + return; + } + thread::sleep(Duration::from_millis(5)); + } + panic!("EDT export marker was not observed"); + }); + + let failure = run_build(&config, &build_args(false)) + .expect_err("deferred EDT commit must detect concurrent state"); + concurrent_commit.join().expect("concurrent commit thread"); + let result = failure.payload.expect("structured failure"); + let failed_step = result.steps.last().expect("failed EDT commit step"); + let receipt = serde_json::to_value(&failed_step.receipt).expect("failed EDT receipt"); + + assert!(failure + .error + .message() + .contains("concurrent state modification")); + assert!(!failed_step.ok); + assert!(matches!(failed_step.mode, BuildMode::EdtExport)); + assert_eq!(receipt["status"], "failed"); + assert!(!receipt["requested"] + .as_array() + .expect("requested") + .is_empty()); + assert_eq!(receipt["processed"], serde_json::json!([])); + let designer_was_loaded = designer_calls.exists() + && fs::read_to_string(&designer_calls) + .expect("Designer calls") + .contains("/LoadConfigFromFiles"); + assert_eq!(designer_was_loaded, !generated_output_is_stable); + } } #[cfg(unix)] @@ -2602,6 +3396,230 @@ mod tests { assert!(!calls.exists()); } + #[cfg(unix)] + #[test] + fn designer_build_uses_private_stage_and_preserves_source_cdfi() { + let dir = tempdir().expect("tempdir"); + let base = dir.path().join("base"); + let work = dir.path().join("work"); + let script = dir.path().join("1cv8"); + let calls = dir.path().join("calls.log"); + create_source_tree(&base); + let source_cdfi = base.join("main/ConfigDumpInfo.xml"); + fs::write(&source_cdfi, b"user-owned sentinel").expect("source CDFI"); + let source_before = fs::read(&source_cdfi).expect("source before"); + write_designer_script(&script, &calls, None); + let config = build_config( + &base, + &work, + &script, + 20, + SourceFormat::Designer, + BuilderBackend::Designer, + ); + prime_snapshots(&config); + + let result = run_build(&config, &build_args(true)).expect("build"); + let calls = fs::read_to_string(calls).expect("calls"); + let main_context = SourceSetsService::new(&config) + .expect("service") + .designer_contexts() + .expect("contexts") + .into_iter() + .find(|context| context.name() == "main") + .expect("main context"); + + assert!(result.ok); + assert!(calls.contains("/ib-state/v1/")); + assert!(calls.contains("/transactions/designer-build-")); + assert!(calls.contains("seeded=no")); + assert!(!calls.contains(base.join("main").display().to_string().as_str())); + assert_eq!(fs::read(source_cdfi).expect("source after"), source_before); + assert!(matches!( + crate::use_cases::runtime_state::inspect_private_cdfi( + &main_context.private_cdfi_path() + ) + .expect("private CDFI"), + crate::use_cases::runtime_state::PrivateCdfiState::Valid(_) + )); + } + + #[cfg(unix)] + #[test] + fn missing_or_corrupt_private_cdfi_forces_full_designer_load() { + let dir = tempdir().expect("tempdir"); + let base = dir.path().join("base"); + let work = dir.path().join("work"); + let script = dir.path().join("1cv8"); + let calls = dir.path().join("calls.log"); + create_source_tree(&base); + write_designer_script(&script, &calls, None); + let config = build_config( + &base, + &work, + &script, + 20, + SourceFormat::Designer, + BuilderBackend::Designer, + ); + prime_snapshots(&config); + let main_context = SourceSetsService::new(&config) + .expect("service") + .designer_contexts() + .expect("contexts") + .into_iter() + .find(|context| context.name() == "main") + .expect("main context"); + + fs::remove_file(main_context.private_cdfi_path()).expect("remove CDFI"); + let first = run_build(&config, &build_args(false)).expect("missing CDFI build"); + assert!(matches!(first.steps[0].mode, BuildMode::Full)); + assert!(!fs::read_to_string(&calls) + .expect("first calls") + .contains("-partial")); + + fs::write(main_context.private_cdfi_path(), "").expect("corrupt CDFI"); + remove_file_if_exists(&calls); + let second = run_build(&config, &build_args(false)).expect("corrupt CDFI build"); + assert!(matches!(second.steps[0].mode, BuildMode::Full)); + assert!(!fs::read_to_string(calls) + .expect("second calls") + .contains("-partial")); + } + + #[cfg(unix)] + #[test] + fn seeded_designer_failure_is_not_retried_and_preserves_private_state() { + let dir = tempdir().expect("tempdir"); + let base = dir.path().join("base"); + let work = dir.path().join("work"); + let script = dir.path().join("1cv8"); + let calls = dir.path().join("calls.log"); + create_source_tree(&base); + write_designer_script(&script, &calls, Some("/LoadConfigFromFiles")); + let config = build_config( + &base, + &work, + &script, + 20, + SourceFormat::Designer, + BuilderBackend::Designer, + ); + prime_snapshots(&config); + let main_context = SourceSetsService::new(&config) + .expect("service") + .designer_contexts() + .expect("contexts") + .into_iter() + .find(|context| context.name() == "main") + .expect("main context"); + let old_cdfi = fs::read(main_context.private_cdfi_path()).expect("old CDFI"); + let old_generation = storage_generation(&config, "main"); + fs::write(base.join("main/Catalogs.Items/ObjectModule.bsl"), "changed") + .expect("change source"); + + let error = run_build(&config, &build_args(false)).expect_err("load failure"); + let calls_text = fs::read_to_string(&calls).expect("calls"); + + assert!(error.error.message().contains("load")); + assert_eq!(calls_text.matches("/LoadConfigFromFiles").count(), 1); + assert!(calls_text.contains("-partial")); + assert!(calls_text.contains("seeded=yes")); + assert_eq!( + fs::read(main_context.private_cdfi_path()).expect("preserved CDFI"), + old_cdfi + ); + assert_eq!(storage_generation(&config, "main"), old_generation); + + write_designer_script(&script, &calls, None); + remove_file_if_exists(&calls); + let recovery = run_build(&config, &build_args(false)).expect("safe recovery build"); + let recovery_calls = fs::read_to_string(calls).expect("recovery calls"); + assert!(matches!(recovery.steps[0].mode, BuildMode::Full)); + assert!(recovery_calls.contains("seeded=no")); + assert!(!recovery_calls.contains("-partial")); + } + + #[cfg(unix)] + #[test] + fn designer_load_is_not_started_when_in_progress_marker_cannot_be_persisted() { + let dir = tempdir().expect("tempdir"); + let base = dir.path().join("base"); + let work = dir.path().join("work"); + let script = dir.path().join("1cv8"); + let calls = dir.path().join("calls.log"); + create_source_tree(&base); + write_designer_script(&script, &calls, None); + let config = build_config( + &base, + &work, + &script, + 20, + SourceFormat::Designer, + BuilderBackend::Designer, + ); + prime_snapshots(&config); + let main_context = SourceSetsService::new(&config) + .expect("service") + .designer_contexts() + .expect("contexts") + .into_iter() + .find(|context| context.name() == "main") + .expect("main context"); + let marker = main_context + .transactions_dir() + .parent() + .expect("state dir") + .join("full-rebuild-required"); + fs::create_dir(&marker).expect("blocking marker directory"); + fs::write(base.join("main/Catalogs.Items/ObjectModule.bsl"), "changed") + .expect("change source"); + remove_file_if_exists(&calls); + + let _failure = run_build(&config, &build_args(false)).expect_err("marker failure"); + + assert!( + !calls.exists(), + "Designer must not start before marker fsync" + ); + } + + #[cfg(unix)] + #[test] + fn failed_explicit_full_load_forces_next_normal_load_unseeded() { + let dir = tempdir().expect("tempdir"); + let base = dir.path().join("base"); + let work = dir.path().join("work"); + let script = dir.path().join("1cv8"); + let calls = dir.path().join("calls.log"); + create_source_tree(&base); + write_designer_script(&script, &calls, Some("/LoadConfigFromFiles")); + let config = build_config( + &base, + &work, + &script, + 20, + SourceFormat::Designer, + BuilderBackend::Designer, + ); + prime_snapshots(&config); + fs::write(base.join("main/Catalogs.Items/ObjectModule.bsl"), "changed") + .expect("change source"); + + run_build(&config, &build_args(true)).expect_err("explicit full load failure"); + assert!(fs::read_to_string(&calls) + .expect("failed calls") + .contains("seeded=no")); + + write_designer_script(&script, &calls, None); + remove_file_if_exists(&calls); + let recovery = run_build(&config, &build_args(false)).expect("normal recovery"); + let recovery_calls = fs::read_to_string(&calls).expect("recovery calls"); + assert!(matches!(recovery.steps[0].mode, BuildMode::Full)); + assert!(recovery_calls.contains("seeded=no")); + assert!(!recovery_calls.contains("-partial")); + } + #[cfg(unix)] #[test] fn changed_configuration_runs_partial_load_and_commits_state() { @@ -2635,6 +3653,24 @@ mod tests { assert!(matches!(result.steps[0].mode, BuildMode::Partial { .. })); assert!(result.steps[0].ok); + let receipt = serde_json::to_value(&result.steps[0].receipt).expect("receipt"); + assert_eq!(receipt["status"], "applied"); + assert_eq!(receipt["requested"].as_array().expect("requested").len(), 1); + assert_eq!( + receipt["requested"][0]["path"], + "Catalogs.Items/ObjectModule.bsl" + ); + assert_ne!( + receipt["requested"][0]["preHash"], + receipt["requested"][0]["postHash"] + ); + let descriptor = receipt["processed"] + .as_array() + .expect("processed") + .iter() + .find(|target| target["path"] == "Catalogs.Items/ObjectModule.xml") + .expect("closure descriptor"); + assert_eq!(descriptor["preHash"], descriptor["postHash"]); assert!(calls_text.contains("/LoadConfigFromFiles")); assert!(calls_text.contains("-partial")); assert!(calls_text.contains("/UpdateDBCfg")); @@ -2643,6 +3679,103 @@ mod tests { let rerun = run_build(&config, &build_args(false)).expect("rerun"); assert!(matches!(rerun.steps[0].mode, BuildMode::Skipped)); + let skipped = serde_json::to_value(&rerun.steps[0].receipt).expect("receipt"); + assert_eq!(skipped["status"], "skipped"); + assert_eq!(skipped["processed"], serde_json::json!([])); + } + + #[cfg(unix)] + #[test] + fn full_load_receipt_keeps_deleted_target_with_absent_post_hash() { + let dir = tempdir().expect("tempdir"); + let base = dir.path().join("base"); + let work = dir.path().join("work"); + let script = dir.path().join("1cv8"); + let calls = dir.path().join("calls.log"); + create_source_tree(&base); + write_designer_script(&script, &calls, None); + let config = build_config( + &base, + &work, + &script, + 20, + SourceFormat::Designer, + BuilderBackend::Designer, + ); + prime_snapshots(&config); + fs::remove_file( + base.join("main") + .join("Catalogs.Items") + .join("ObjectModule.bsl"), + ) + .expect("delete module"); + + let result = run_build(&config, &build_args(false)).expect("build"); + let receipt = serde_json::to_value(&result.steps[0].receipt).expect("receipt"); + let deleted = receipt["processed"] + .as_array() + .expect("processed") + .iter() + .find(|target| target["path"] == "Catalogs.Items/ObjectModule.bsl") + .expect("deleted target"); + + assert!(matches!(result.steps[0].mode, BuildMode::Full)); + assert!(deleted["preHash"].is_string()); + assert!(deleted["postHash"].is_null()); + } + + #[test] + fn full_plan_commits_exact_pre_platform_observation_and_detects_later_edit() { + let dir = tempdir().expect("tempdir"); + let base = dir.path().join("base"); + let work = dir.path().join("work"); + let script = dir.path().join("1cv8"); + create_source_tree(&base); + let config = build_config( + &base, + &work, + &script, + 20, + SourceFormat::Designer, + BuilderBackend::Designer, + ); + let source_set = &config.source_sets[0]; + let context = SourceSetsService::new(&config) + .expect("service") + .designer_contexts() + .expect("contexts") + .into_iter() + .find(|candidate| candidate.name() == source_set.name) + .expect("context"); + + let plan = + plan_configurator_load_step(source_set, &context, true, None, 20).expect("full plan"); + let receipt = serde_json::to_value(receipt_after_success(&plan)).expect("receipt"); + let observed = receipt["processed"] + .as_array() + .expect("processed") + .iter() + .find(|target| target["path"] == ".project") + .expect("project target")["postHash"] + .as_str() + .expect("post hash") + .to_owned(); + + fs::write(base.join("main").join(".project"), "edited after planning") + .expect("concurrent edit"); + let StepPlan::Execute { commit, .. } = &plan else { + panic!("full plan must execute"); + }; + commit_step_state(source_set, &context, commit).expect("commit observation"); + + let stored = HashStorage::new(context.storage_path()) + .load_snapshot() + .expect("snapshot"); + assert_eq!(stored.entries[".project"].hash, observed); + assert!(matches!( + analyzer::analyze_context(&context).outcome, + Ok(AnalysisOutcome::Changes { .. }) + )); } #[cfg(unix)] @@ -2826,13 +3959,14 @@ mod tests { ); prime_snapshots(&config); - let service = SourceSetsService::new(&config); + let service = SourceSetsService::new(&config).expect("service"); let main_context = service .designer_contexts() + .expect("contexts") .into_iter() .find(|context| context.name() == "main") .expect("main context"); - fs::write(main_context.storage_path(&config.work_path), "corrupt").expect("corrupt main"); + fs::write(main_context.storage_path(), "corrupt").expect("corrupt main"); fs::write( base.join("ext").join("CommonModules").join("Module.bsl"), "procedure Test()\n // ext changed\nendprocedure", @@ -2962,9 +4096,9 @@ mod tests { ); prime_snapshots(&config); - let service = SourceSetsService::new(&config); - for context in service.designer_contexts() { - let storage_path = context.storage_path(&config.work_path); + let service = SourceSetsService::new(&config).expect("service"); + for context in service.designer_contexts().expect("contexts") { + let storage_path = context.storage_path(); fs::write(storage_path, "corrupt").expect("corrupt storage"); } @@ -2999,13 +4133,14 @@ mod tests { ); prime_snapshots(&config); - let service = SourceSetsService::new(&config); + let service = SourceSetsService::new(&config).expect("service"); let main_context = service .designer_contexts() + .expect("contexts") .into_iter() .find(|context| context.name() == "main") .expect("main context"); - let storage_path = main_context.storage_path(&config.work_path); + let storage_path = main_context.storage_path(); std::fs::remove_file(&storage_path).expect("remove storage file"); std::fs::create_dir_all(&storage_path).expect("replace with directory"); @@ -3057,6 +4192,19 @@ mod tests { assert!(result.steps[0].ok); assert!(!result.steps[1].ok); + let applied = serde_json::to_value(&result.steps[0].receipt).expect("applied receipt"); + let failed = serde_json::to_value(&result.steps[1].receipt).expect("failed receipt"); + assert_eq!(applied["status"], "applied"); + assert!(!applied["processed"] + .as_array() + .expect("processed") + .is_empty()); + assert_eq!(failed["status"], "failed"); + assert!(!failed["requested"] + .as_array() + .expect("requested") + .is_empty()); + assert_eq!(failed["processed"], serde_json::json!([])); assert!(result.steps[1] .message .as_deref() @@ -3078,6 +4226,7 @@ mod tests { ok: true, message: Some("forced full rebuild".to_owned()), duration_ms: 1, + receipt: Default::default(), }, crate::domain::build::BuildStep { source_set: "ext".to_owned(), @@ -3085,6 +4234,7 @@ mod tests { ok: false, message: Some("aborted after previous failure".to_owned()), duration_ms: 0, + receipt: Default::default(), }, ], duration_ms: 42, diff --git a/src/use_cases/build_project/coordinator.rs b/src/use_cases/build_project/coordinator.rs index 4f06468..bacb23a 100644 --- a/src/use_cases/build_project/coordinator.rs +++ b/src/use_cases/build_project/coordinator.rs @@ -1,5 +1,36 @@ use super::*; +#[must_use = "EDT state must remain pending until the downstream generated stage succeeds"] +struct PendingEdtCommit { + context: SourceSetContext, + commit: StepCommit, + receipt: PlannedReceipt, +} + +impl PendingEdtCommit { + fn new(context: SourceSetContext, commit: StepCommit, receipt: PlannedReceipt) -> Self { + Self { + context, + commit, + receipt, + } + } + + fn commit(self, source_set: &SourceSetConfig) -> Result<(), PendingEdtCommitFailure> { + commit_step_state(source_set, &self.context, &self.commit).map_err(|error| { + PendingEdtCommitFailure { + error, + receipt: self.receipt, + } + }) + } +} + +struct PendingEdtCommitFailure { + error: AppError, + receipt: PlannedReceipt, +} + pub(super) fn run_build_designer( context: &ExecutionContext, config: &AppConfig, @@ -12,7 +43,16 @@ pub(super) fn run_build_designer( ); let started = Instant::now(); - let inventory = SourceSetInventory::new(config); + let inventory = SourceSetInventory::new(config).map_err(|error| { + BuildExecutionFailure::with_payload( + AppError::from(error), + BuildResult { + ok: false, + steps: vec![], + duration_ms: started.elapsed().as_millis() as u64, + }, + ) + })?; let ordered_source_sets = match selected_ordered_source_sets(&inventory, args.source_set.as_deref()) { Ok(source_sets) => source_sets, @@ -30,6 +70,19 @@ pub(super) fn run_build_designer( let selected_designer_contexts = designer_contexts_for_source_sets(&inventory, &ordered_source_sets); + for source_context in &selected_designer_contexts { + recover_designer_state(source_context).map_err(|error| { + BuildExecutionFailure::with_payload( + AppError::Runtime(error.to_string()), + BuildResult { + ok: false, + steps: vec![], + duration_ms: started.elapsed().as_millis() as u64, + }, + ) + })?; + } + let analysis_by_name = if args.full_rebuild { None } else { @@ -83,10 +136,26 @@ pub(super) fn run_build_designer( continue; } + let cdfi_plan = match resolve_designer_cdfi_plan(&source_context, args.full_rebuild) { + Ok(plan) => plan, + Err(error) => { + let result = fail_from_source_set_index( + started, + steps, + &ordered_source_sets, + index, + source_set, + BuildMode::Skipped, + error.to_string(), + ); + return Err(BuildExecutionFailure::with_payload(error, result)); + } + }; + let effective_full_rebuild = cdfi_plan.forces_full_rebuild(); let plan = match plan_configurator_load_step( source_set, &source_context, - args.full_rebuild, + effective_full_rebuild, analysis_by_name.as_ref(), config.build.partial_load_threshold, ) { @@ -109,19 +178,24 @@ pub(super) fn run_build_designer( }; match plan { - StepPlan::Skip { message, ok } => { + StepPlan::Skip { + message, + ok, + receipt, + } => { debug!( source_set = source_set.name.as_str(), message = message.as_str(), "skipping build step" ); - push_build_step( + push_build_step_with_receipt( &mut steps, &source_set.name, BuildMode::Skipped, ok, message, 0, + receipt, ) } StepPlan::Execute { @@ -129,6 +203,7 @@ pub(super) fn run_build_designer( message, partial_paths, commit, + receipt, } => { debug!( source_set = source_set.name.as_str(), @@ -151,6 +226,8 @@ pub(super) fn run_build_designer( mode.clone(), error.to_string(), ); + let result = + attach_failed_receipt(result, &source_set.name, &receipt); return Err(BuildExecutionFailure::with_payload( AppError::from(error), result, @@ -174,14 +251,16 @@ pub(super) fn run_build_designer( index, partial_paths.as_deref(), &commit, + &cdfi_plan, ) { - Ok(warnings) => push_build_step( + Ok(warnings) => push_build_step_with_receipt( &mut steps, &source_set.name, mode, true, merge_step_message(message, &warnings), step_started.elapsed().as_millis() as u64, + receipt.applied(), ), Err(error) => { let result = fail_from_source_set_index( @@ -193,6 +272,7 @@ pub(super) fn run_build_designer( mode, error.to_string(), ); + let result = attach_failed_receipt(result, &source_set.name, &receipt); return Err(BuildExecutionFailure::with_payload(error, result)); } } @@ -219,7 +299,16 @@ pub(super) fn run_build_ibcmd( ); let started = Instant::now(); - let inventory = SourceSetInventory::new(config); + let inventory = SourceSetInventory::new(config).map_err(|error| { + BuildExecutionFailure::with_payload( + AppError::from(error), + BuildResult { + ok: false, + steps: vec![], + duration_ms: started.elapsed().as_millis() as u64, + }, + ) + })?; let ordered_source_sets = match selected_ordered_source_sets(&inventory, args.source_set.as_deref()) { Ok(source_sets) => source_sets, @@ -281,19 +370,24 @@ pub(super) fn run_build_ibcmd( }; match plan { - StepPlan::Skip { message, ok } => { + StepPlan::Skip { + message, + ok, + receipt, + } => { debug!( source_set = source_set.name.as_str(), message = message.as_str(), "skipping build step" ); - push_build_step( + push_build_step_with_receipt( &mut steps, &source_set.name, BuildMode::Skipped, ok, message, 0, + receipt, ) } StepPlan::Execute { @@ -301,6 +395,7 @@ pub(super) fn run_build_ibcmd( message, partial_paths, commit, + receipt, } => { debug!( source_set = source_set.name.as_str(), @@ -323,6 +418,8 @@ pub(super) fn run_build_ibcmd( mode.clone(), error.to_string(), ); + let result = + attach_failed_receipt(result, &source_set.name, &receipt); return Err(BuildExecutionFailure::with_payload( AppError::from(error), result, @@ -346,13 +443,14 @@ pub(super) fn run_build_ibcmd( partial_paths.as_deref(), &commit, ) { - Ok(warnings) => push_build_step( + Ok(warnings) => push_build_step_with_receipt( &mut steps, &source_set.name, mode, true, merge_step_message(message, &warnings), step_started.elapsed().as_millis() as u64, + receipt.applied(), ), Err(error) => { let result = fail_from_source_set_index( @@ -364,6 +462,7 @@ pub(super) fn run_build_ibcmd( mode, error.to_string(), ); + let result = attach_failed_receipt(result, &source_set.name, &receipt); return Err(BuildExecutionFailure::with_payload(error, result)); } } @@ -400,7 +499,16 @@ pub(super) fn run_build_edt( } let started = Instant::now(); - let inventory = SourceSetInventory::new(config); + let inventory = SourceSetInventory::new(config).map_err(|error| { + BuildExecutionFailure::with_payload( + AppError::from(error), + BuildResult { + ok: false, + steps: vec![], + duration_ms: started.elapsed().as_millis() as u64, + }, + ) + })?; let ordered_source_sets = match selected_ordered_source_sets(&inventory, args.source_set.as_deref()) { Ok(source_sets) => source_sets, @@ -437,9 +545,11 @@ pub(super) fn run_build_edt( let Some(designer_context) = inventory.designer_context(&source_set.name).cloned() else { continue; }; + let mut pending_edt_commit = None; let edt_stage = match plan_edt_export_step( source_set, + &edt_context, args.full_rebuild, edt_analysis_by_name.as_ref(), ) { @@ -477,6 +587,8 @@ pub(super) fn run_build_edt( BuildMode::EdtExport, error.to_string(), ); + let result = + attach_failed_plan_receipt(result, &source_set.name, &edt_stage); return Err(BuildExecutionFailure::with_payload( AppError::from(error), result, @@ -504,6 +616,7 @@ pub(super) fn run_build_edt( BuildMode::EdtExport, error.to_string(), ); + let result = attach_failed_plan_receipt(result, &source_set.name, &edt_stage); return Err(BuildExecutionFailure::with_payload(error, result)); } log_timeline_stage( @@ -541,6 +654,11 @@ pub(super) fn run_build_edt( BuildMode::EdtExport, app_error.to_string(), ); + let result = attach_failed_plan_receipt( + result, + &source_set.name, + &edt_stage, + ); return Err(BuildExecutionFailure::with_payload( app_error, result, )); @@ -557,6 +675,11 @@ pub(super) fn run_build_edt( BuildMode::EdtExport, app_error.to_string(), ); + let result = attach_failed_plan_receipt( + result, + &source_set.name, + &edt_stage, + ); return Err(BuildExecutionFailure::with_payload(app_error, result)); } }, @@ -582,12 +705,9 @@ pub(super) fn run_build_edt( Ok(descriptors) => { match &edt_stage { StepPlan::Execute { commit, .. } => { - if let Err(app_error) = commit_step_state( - source_set, - &edt_context, - &config.work_path, - commit, - ) { + if let Err(app_error) = + commit_step_state(source_set, &edt_context, commit) + { let result = fail_from_source_set_index( started, steps, @@ -597,12 +717,18 @@ pub(super) fn run_build_edt( BuildMode::EdtExport, app_error.to_string(), ); + let result = attach_failed_plan_receipt( + result, + &source_set.name, + &edt_stage, + ); return Err(BuildExecutionFailure::with_payload(app_error, result)); } } StepPlan::Skip { .. } => {} } - push_build_step( + let receipt = receipt_after_success(&edt_stage); + push_build_step_with_receipt( &mut steps, &source_set.name, BuildMode::EdtExport, @@ -615,6 +741,7 @@ pub(super) fn run_build_edt( &[], ), export_started.elapsed().as_millis() as u64, + receipt, ) } Err(error) => { @@ -627,6 +754,7 @@ pub(super) fn run_build_edt( BuildMode::EdtExport, error.to_string(), ); + let result = attach_failed_plan_receipt(result, &source_set.name, &edt_stage); return Err(BuildExecutionFailure::with_payload(error, result)); } } @@ -636,14 +764,19 @@ pub(super) fn run_build_edt( let edt_stage_skipped = matches!(&edt_stage, StepPlan::Skip { .. }); match edt_stage { - StepPlan::Skip { message, ok } => { - push_build_step( + StepPlan::Skip { + message, + ok, + receipt, + } => { + push_build_step_with_receipt( &mut steps, &source_set.name, BuildMode::Skipped, ok, message, 0, + receipt, ); } StepPlan::Execute { @@ -651,6 +784,7 @@ pub(super) fn run_build_edt( partial_paths: _, commit, mode: _, + receipt, } => { let edt = match edt_binary.clone() { Some(path) => path, @@ -667,6 +801,8 @@ pub(super) fn run_build_edt( BuildMode::EdtExport, error.to_string(), ); + let result = + attach_failed_receipt(result, &source_set.name, &receipt); return Err(BuildExecutionFailure::with_payload( AppError::from(error), result, @@ -714,6 +850,11 @@ pub(super) fn run_build_edt( BuildMode::EdtExport, app_error.to_string(), ); + let result = attach_failed_receipt( + result, + &source_set.name, + &receipt, + ); return Err(BuildExecutionFailure::with_payload( app_error, result, )); @@ -730,6 +871,8 @@ pub(super) fn run_build_edt( BuildMode::EdtExport, app_error.to_string(), ); + let result = + attach_failed_receipt(result, &source_set.name, &receipt); return Err(BuildExecutionFailure::with_payload( app_error, result, )); @@ -777,42 +920,56 @@ pub(super) fn run_build_edt( BuildMode::EdtExport, error.to_string(), ); + let result = attach_failed_receipt(result, &source_set.name, &receipt); return Err(BuildExecutionFailure::with_payload(error, result)); } }; - if let Err(app_error) = - commit_step_state(source_set, &edt_context, &config.work_path, &commit) - { - let result = fail_from_source_set_index( - started, - steps, - &ordered_source_sets, - index, - source_set, - BuildMode::EdtExport, - app_error.to_string(), - ); - return Err(BuildExecutionFailure::with_payload(app_error, result)); - } + pending_edt_commit = Some(PendingEdtCommit::new( + edt_context.clone(), + commit, + receipt.clone(), + )); - push_build_step( + push_build_step_with_receipt( &mut steps, &source_set.name, BuildMode::EdtExport, true, merge_step_message("EDT export completed".to_owned(), &export_warnings), export_started.elapsed().as_millis() as u64, + receipt.applied(), ); } } + let designer_cdfi_plan = if config.builder == BuilderBackend::Designer { + match resolve_designer_cdfi_plan(&designer_context, args.full_rebuild) { + Ok(plan) => plan, + Err(error) => { + let result = fail_from_source_set_index( + started, + steps, + &ordered_source_sets, + index, + source_set, + BuildMode::Skipped, + error.to_string(), + ); + return Err(BuildExecutionFailure::with_payload(error, result)); + } + } + } else if args.full_rebuild { + DesignerCdfiPlan::IbcmdFull + } else { + DesignerCdfiPlan::IbcmdNormal + }; + let designer_effective_full_rebuild = designer_cdfi_plan.forces_full_rebuild(); let designer_stage = match plan_generated_designer_load_step( source_set, &designer_context, - args.full_rebuild, + designer_effective_full_rebuild, edt_stage_skipped, config.build.partial_load_threshold, - &config.work_path, ) { Ok(plan) => plan, Err(error) => { @@ -833,14 +990,35 @@ pub(super) fn run_build_edt( }; match designer_stage { - StepPlan::Skip { message, ok } => { - push_build_step( + StepPlan::Skip { + message, + ok, + receipt, + } => { + if let Some(pending) = pending_edt_commit.take() { + if let Err(failure) = pending.commit(source_set) { + let result = fail_from_source_set_index( + started, + steps, + &ordered_source_sets, + index, + source_set, + BuildMode::EdtExport, + failure.error.to_string(), + ); + let result = + attach_failed_receipt(result, &source_set.name, &failure.receipt); + return Err(BuildExecutionFailure::with_payload(failure.error, result)); + } + } + push_build_step_with_receipt( &mut steps, &source_set.name, BuildMode::Skipped, ok, message, 0, + receipt, ); } StepPlan::Execute { @@ -848,6 +1026,7 @@ pub(super) fn run_build_edt( message, partial_paths, commit, + receipt, } => { let load_started = Instant::now(); let load_result = match config.builder { @@ -867,6 +1046,11 @@ pub(super) fn run_build_edt( mode.clone(), error.to_string(), ); + let result = attach_failed_receipt( + result, + &source_set.name, + &receipt, + ); return Err(BuildExecutionFailure::with_payload( AppError::from(error), result, @@ -888,6 +1072,7 @@ pub(super) fn run_build_edt( index, partial_paths.as_deref(), &commit, + &designer_cdfi_plan, ) } BuilderBackend::Ibcmd => { @@ -906,6 +1091,11 @@ pub(super) fn run_build_edt( mode.clone(), error.to_string(), ); + let result = attach_failed_receipt( + result, + &source_set.name, + &receipt, + ); return Err(BuildExecutionFailure::with_payload( AppError::from(error), result, @@ -930,14 +1120,39 @@ pub(super) fn run_build_edt( } }; match load_result { - Ok(warnings) => push_build_step( - &mut steps, - &source_set.name, - mode, - true, - merge_step_message(message, &warnings), - load_started.elapsed().as_millis() as u64, - ), + Ok(warnings) => { + if let Some(pending) = pending_edt_commit.take() { + if let Err(failure) = pending.commit(source_set) { + let result = fail_from_source_set_index( + started, + steps, + &ordered_source_sets, + index, + source_set, + BuildMode::EdtExport, + failure.error.to_string(), + ); + let result = attach_failed_receipt( + result, + &source_set.name, + &failure.receipt, + ); + return Err(BuildExecutionFailure::with_payload( + failure.error, + result, + )); + } + } + push_build_step_with_receipt( + &mut steps, + &source_set.name, + mode, + true, + merge_step_message(message, &warnings), + load_started.elapsed().as_millis() as u64, + receipt.applied(), + ) + } Err(error) => { let result = fail_from_source_set_index( started, @@ -948,6 +1163,7 @@ pub(super) fn run_build_edt( mode, error.to_string(), ); + let result = attach_failed_receipt(result, &source_set.name, &receipt); return Err(BuildExecutionFailure::with_payload(error, result)); } } diff --git a/src/use_cases/build_project/helpers.rs b/src/use_cases/build_project/helpers.rs index f6f1134..1e375d7 100644 --- a/src/use_cases/build_project/helpers.rs +++ b/src/use_cases/build_project/helpers.rs @@ -3,11 +3,11 @@ use std::path::{Path, PathBuf}; use std::time::Instant; use crate::change_detection::analyzer::{self, AnalysisOutcome, PreparedStateUpdate}; -use crate::change_detection::hash_storage::{HashStorage, StorageError}; use crate::change_detection::partial_load::{self, LoadDecision}; use crate::config::model::{AppConfig, SourceSetConfig, SourceSetPurpose}; use crate::domain::build::{BuildMode, BuildResult, BuildStep}; use crate::domain::source_set::SourceSetContext; +use crate::domain::sync_receipt::{SyncReceipt, SyncTarget}; use crate::platform::designer::DesignerDsl; use crate::platform::ibcmd::{IbcmdConnection, IbcmdDsl, IbcmdError}; use crate::platform::process::ProcessRunner; @@ -26,22 +26,214 @@ pub(super) type AnalysisByName = pub(super) enum StepCommit { Prepared(PreparedStateUpdate), - RescanFull { recover_storage: bool }, + FullObservation(PreparedStateUpdate), +} + +impl StepCommit { + pub(super) fn prepared_state(&self) -> &PreparedStateUpdate { + match self { + Self::Prepared(prepared) | Self::FullObservation(prepared) => prepared, + } + } } pub(super) enum StepPlan { Skip { message: String, ok: bool, + receipt: SyncReceipt, }, Execute { mode: BuildMode, message: String, partial_paths: Option>, commit: StepCommit, + receipt: PlannedReceipt, }, } +#[derive(Clone)] +pub(super) struct PlannedReceipt { + applied: SyncReceipt, + failed: SyncReceipt, +} + +impl PlannedReceipt { + fn new( + source_set: &SourceSetConfig, + requested: Vec, + processed: Vec, + ) -> Result { + let failed = SyncReceipt::failed(requested.clone()).map_err(|error| { + analyzer::ChangeDetectionError::InvalidInventory { + source_set: source_set.name.clone(), + reason: error.to_string(), + } + })?; + let applied = SyncReceipt::applied(requested, processed, vec![]).map_err(|error| { + analyzer::ChangeDetectionError::InvalidInventory { + source_set: source_set.name.clone(), + reason: error.to_string(), + } + })?; + Ok(Self { applied, failed }) + } + + pub fn applied(&self) -> SyncReceipt { + self.applied.clone() + } + + pub fn failed(&self) -> SyncReceipt { + self.failed.clone() + } +} + +fn sync_target_from_change( + source_set: &SourceSetConfig, + change: &analyzer::FileChange, +) -> Result { + SyncTarget::new( + &change.rel_path, + change.pre_hash.clone(), + change.post_hash.clone(), + ) + .map_err(|error| analyzer::ChangeDetectionError::InvalidInventory { + source_set: source_set.name.clone(), + reason: error.to_string(), + }) +} + +fn planned_full_receipt( + source_set: &SourceSetConfig, + context: &SourceSetContext, +) -> Result<(PlannedReceipt, PreparedStateUpdate), analyzer::ChangeDetectionError> { + let inventory = analyzer::managed_inventory(context)?; + let requested = inventory + .requested + .iter() + .map(|change| sync_target_from_change(source_set, change)) + .collect::, _>>()?; + let requested_by_path = unique_target_index(source_set, &requested)?; + let mut processed = inventory + .current + .iter() + .map(|file| { + let pre_hash = match requested_by_path.get(file.rel_path.as_str()) { + Some(target) => target.pre_hash().map(str::to_owned), + None => Some(file.hash.clone()), + }; + SyncTarget::new(&file.rel_path, pre_hash, Some(file.hash.clone())).map_err(|error| { + analyzer::ChangeDetectionError::InvalidInventory { + source_set: source_set.name.clone(), + reason: error.to_string(), + } + }) + }) + .collect::, _>>()?; + processed.extend( + requested + .iter() + .filter(|target| target.post_hash().is_none()) + .cloned(), + ); + Ok(( + PlannedReceipt::new(source_set, requested, processed)?, + inventory.prepared, + )) +} + +fn planned_change_receipt( + source_set: &SourceSetConfig, + context_path: &Path, + changes: &[analyzer::FileChange], + prepared: &PreparedStateUpdate, + processed_paths: Option<&[PathBuf]>, +) -> Result { + let requested = changes + .iter() + .map(|change| sync_target_from_change(source_set, change)) + .collect::, _>>()?; + let requested_by_path = unique_target_index(source_set, &requested)?; + let selected = match processed_paths { + Some(paths) => partial_load::relative_paths(paths, context_path) + .map_err(|error| analyzer::ChangeDetectionError::InvalidInventory { + source_set: source_set.name.clone(), + reason: error.to_string(), + })? + .into_iter() + .map(|path| { + crate::change_detection::scanner::portable_relative_path( + context_path, + &context_path.join(path), + ) + .map_err(|error| { + analyzer::ChangeDetectionError::InvalidInventory { + source_set: source_set.name.clone(), + reason: error.to_string(), + } + }) + }) + .collect::, _>>()?, + None => prepared + .snapshot + .iter() + .map(|file| file.rel_path.clone()) + .collect(), + }; + let prepared_by_path = prepared + .snapshot + .iter() + .map(|file| (file.rel_path.as_str(), file)) + .collect::>(); + let mut processed = selected + .into_iter() + .map(|rel_path| { + let post_hash = prepared_by_path + .get(rel_path.as_str()) + .map(|file| file.hash.clone()) + .ok_or_else(|| analyzer::ChangeDetectionError::InvalidInventory { + source_set: source_set.name.clone(), + reason: format!("processed path '{rel_path}' is absent from managed inventory"), + })?; + let pre_hash = match requested_by_path.get(rel_path.as_str()) { + Some(target) => target.pre_hash().map(str::to_owned), + None => Some(post_hash.clone()), + }; + SyncTarget::new(rel_path, pre_hash, Some(post_hash)).map_err(|error| { + analyzer::ChangeDetectionError::InvalidInventory { + source_set: source_set.name.clone(), + reason: error.to_string(), + } + }) + }) + .collect::, _>>()?; + if processed_paths.is_none() { + processed.extend( + requested + .iter() + .filter(|target| target.post_hash().is_none()) + .cloned(), + ); + } + PlannedReceipt::new(source_set, requested, processed) +} + +fn unique_target_index<'a>( + source_set: &SourceSetConfig, + targets: &'a [SyncTarget], +) -> Result, analyzer::ChangeDetectionError> { + let mut by_path = HashMap::with_capacity(targets.len()); + for target in targets { + if by_path.insert(target.path(), target).is_some() { + return Err(analyzer::ChangeDetectionError::InvalidInventory { + source_set: source_set.name.clone(), + reason: format!("duplicate normalized path '{}'", target.path()), + }); + } + } + Ok(by_path) +} + pub(super) fn plan_configurator_load_step( source_set: &SourceSetConfig, source_context: &SourceSetContext, @@ -50,13 +242,13 @@ pub(super) fn plan_configurator_load_step( partial_load_threshold: usize, ) -> Result { if full_rebuild { + let (receipt, prepared) = planned_full_receipt(source_set, source_context)?; return Ok(StepPlan::Execute { mode: BuildMode::Full, message: "forced full rebuild".to_owned(), partial_paths: None, - commit: StepCommit::RescanFull { - recover_storage: true, - }, + commit: StepCommit::FullObservation(prepared), + receipt, }); } @@ -64,27 +256,28 @@ pub(super) fn plan_configurator_load_step( .and_then(|analysis| analysis.get(&source_set.name)) .cloned() .expect("every source-set must have an analysis result")?; - Ok(plan_configurator_load_from_analysis( + plan_configurator_load_from_analysis( source_set, - source_context.path(), + source_context, outcome, partial_load_threshold, - )) + ) } pub(super) fn plan_edt_export_step( source_set: &SourceSetConfig, + source_context: &SourceSetContext, full_rebuild: bool, analysis_by_name: Option<&AnalysisByName>, ) -> Result { if full_rebuild { + let (receipt, prepared) = planned_full_receipt(source_set, source_context)?; return Ok(StepPlan::Execute { mode: BuildMode::EdtExport, message: "forced EDT export (--full-rebuild)".to_owned(), partial_paths: None, - commit: StepCommit::RescanFull { - recover_storage: true, - }, + commit: StepCommit::FullObservation(prepared), + receipt, }); } @@ -102,9 +295,11 @@ pub(super) fn plan_edt_export_step( Ok(StepPlan::Skip { message: "no changes".to_owned(), ok: true, + receipt: SyncReceipt::empty_skipped(), }) } - AnalysisOutcome::Fallback => { + AnalysisOutcome::Bootstrap | AnalysisOutcome::Fallback => { + let (receipt, prepared) = planned_full_receipt(source_set, source_context)?; debug!( source_set = source_set.name.as_str(), "edt change analysis result: fallback to full export/load after recoverable issue" @@ -120,18 +315,25 @@ pub(super) fn plan_edt_export_step( message: "fallback to EDT export after recoverable change-detection issue" .to_owned(), partial_paths: None, - commit: StepCommit::RescanFull { - recover_storage: false, - }, + commit: StepCommit::FullObservation(prepared), + receipt, }) } AnalysisOutcome::Changes { changes, prepared } => { log_change_analysis(source_set.name.as_str(), &changes); + let receipt = planned_change_receipt( + source_set, + source_context.path(), + &changes, + &prepared, + None, + )?; Ok(StepPlan::Execute { mode: BuildMode::EdtExport, message: "EDT export after change detection".to_owned(), partial_paths: None, commit: StepCommit::Prepared(prepared), + receipt, }) } } @@ -143,41 +345,41 @@ pub(super) fn plan_generated_designer_load_step( full_rebuild: bool, edt_stage_skipped: bool, partial_load_threshold: usize, - work_path: &Path, ) -> Result { if edt_stage_skipped && !designer_context.path().exists() { return Ok(StepPlan::Skip { message: "no changes".to_owned(), ok: true, + receipt: SyncReceipt::empty_skipped(), }); } if full_rebuild { + let (receipt, prepared) = planned_full_receipt(source_set, designer_context)?; return Ok(StepPlan::Execute { mode: BuildMode::Full, message: "full load from EDT export (--full-rebuild)".to_owned(), partial_paths: None, - commit: StepCommit::RescanFull { - recover_storage: true, - }, + commit: StepCommit::FullObservation(prepared), + receipt, }); } - let outcome = analyzer::analyze_context(designer_context, work_path).outcome?; - Ok(plan_generated_designer_load_from_analysis( + let outcome = analyzer::analyze_context(designer_context).outcome?; + plan_generated_designer_load_from_analysis( source_set, - designer_context.path(), + designer_context, outcome, partial_load_threshold, - )) + ) } fn plan_configurator_load_from_analysis( source_set: &SourceSetConfig, - context_path: &Path, + source_context: &SourceSetContext, outcome: AnalysisOutcome, partial_load_threshold: usize, -) -> StepPlan { +) -> Result { match outcome { AnalysisOutcome::NoChanges => { debug!( @@ -185,12 +387,14 @@ fn plan_configurator_load_from_analysis( found_changes = 0, "change analysis result: found 0 change(s)" ); - StepPlan::Skip { + Ok(StepPlan::Skip { message: "no changes".to_owned(), ok: true, - } + receipt: SyncReceipt::empty_skipped(), + }) } - AnalysisOutcome::Fallback => { + AnalysisOutcome::Bootstrap | AnalysisOutcome::Fallback => { + let (receipt, prepared) = planned_full_receipt(source_set, source_context)?; debug!( source_set = source_set.name.as_str(), "change analysis result: fallback to full load after recoverable issue" @@ -201,21 +405,20 @@ fn plan_configurator_load_from_analysis( "fallback to full load after recoverable issue", TimelineStageStatus::Succeeded, ); - StepPlan::Execute { + Ok(StepPlan::Execute { mode: BuildMode::Full, message: "fallback to full load after recoverable change-detection issue" .to_owned(), partial_paths: None, - commit: StepCommit::RescanFull { - recover_storage: false, - }, - } + commit: StepCommit::FullObservation(prepared), + receipt, + }) } AnalysisOutcome::Changes { changes, prepared } => { log_change_analysis(source_set.name.as_str(), &changes); plan_partial_or_full_load( source_set, - context_path, + source_context.path(), changes, prepared, partial_load_threshold, @@ -227,10 +430,10 @@ fn plan_configurator_load_from_analysis( fn plan_generated_designer_load_from_analysis( source_set: &SourceSetConfig, - context_path: &Path, + source_context: &SourceSetContext, outcome: AnalysisOutcome, partial_load_threshold: usize, -) -> StepPlan { +) -> Result { match outcome { AnalysisOutcome::NoChanges => { debug!( @@ -238,12 +441,14 @@ fn plan_generated_designer_load_from_analysis( found_changes = 0, "generated designer change analysis result: found 0 change(s)" ); - StepPlan::Skip { + Ok(StepPlan::Skip { message: "no changes".to_owned(), ok: true, - } + receipt: SyncReceipt::empty_skipped(), + }) } - AnalysisOutcome::Fallback => { + AnalysisOutcome::Bootstrap | AnalysisOutcome::Fallback => { + let (receipt, prepared) = planned_full_receipt(source_set, source_context)?; debug!( source_set = source_set.name.as_str(), "generated designer change analysis result: fallback to full load after recoverable issue" @@ -254,21 +459,20 @@ fn plan_generated_designer_load_from_analysis( "fallback to full load after recoverable issue", TimelineStageStatus::Succeeded, ); - StepPlan::Execute { + Ok(StepPlan::Execute { mode: BuildMode::Full, message: "fallback to full load after recoverable change-detection issue" .to_owned(), partial_paths: None, - commit: StepCommit::RescanFull { - recover_storage: false, - }, - } + commit: StepCommit::FullObservation(prepared), + receipt, + }) } AnalysisOutcome::Changes { changes, prepared } => { log_change_analysis(source_set.name.as_str(), &changes); plan_partial_or_full_load( source_set, - context_path, + source_context.path(), changes, prepared, partial_load_threshold, @@ -311,7 +515,7 @@ fn plan_partial_or_full_load( prepared: PreparedStateUpdate, partial_load_threshold: usize, source: LoadPlanSource, -) -> StepPlan { +) -> Result { let decision = if source.forces_full_for_extension() && source_set.purpose == SourceSetPurpose::Extension { @@ -326,6 +530,13 @@ fn plan_partial_or_full_load( match decision { LoadDecision::Partial(paths) => { + let receipt = planned_change_receipt( + source_set, + context_path, + &changes, + &prepared, + Some(&paths), + )?; debug!( source_set = source_set.name.as_str(), partial_file_count = paths.len(), @@ -333,23 +544,26 @@ fn plan_partial_or_full_load( "{}", source.partial_log_message() ); - StepPlan::Execute { + Ok(StepPlan::Execute { mode: BuildMode::Partial { file_count: paths.len(), }, message: format!("partial load of {} files", paths.len()), partial_paths: Some(paths), commit: StepCommit::Prepared(prepared), - } + receipt, + }) } LoadDecision::Full => { + let receipt = + planned_change_receipt(source_set, context_path, &changes, &prepared, None)?; debug!( source_set = source_set.name.as_str(), threshold = partial_load_threshold, "{}", source.full_log_message() ); - StepPlan::Execute { + Ok(StepPlan::Execute { mode: BuildMode::Full, message: if source.forces_full_for_extension() && source_set.purpose == SourceSetPurpose::Extension @@ -360,7 +574,8 @@ fn plan_partial_or_full_load( }, partial_paths: None, commit: StepCommit::Prepared(prepared), - } + receipt, + }) } } } @@ -396,6 +611,35 @@ pub(super) fn push_build_step( ok: bool, message: String, duration_ms: u64, +) { + let step = BuildStep { + source_set: source_set_name.to_owned(), + mode: mode.clone(), + ok, + message: Some(message), + duration_ms, + receipt: if ok { + if matches!(mode, BuildMode::Skipped) { + crate::domain::sync_receipt::SyncReceipt::empty_skipped() + } else { + crate::domain::sync_receipt::SyncReceipt::empty_applied() + } + } else { + crate::domain::sync_receipt::SyncReceipt::empty_failed() + }, + }; + log_build_step_timeline(&step); + steps.push(step); +} + +pub(super) fn push_build_step_with_receipt( + steps: &mut Vec, + source_set_name: &str, + mode: BuildMode, + ok: bool, + message: String, + duration_ms: u64, + receipt: SyncReceipt, ) { let step = BuildStep { source_set: source_set_name.to_owned(), @@ -403,37 +647,59 @@ pub(super) fn push_build_step( ok, message: Some(message), duration_ms, + receipt, }; log_build_step_timeline(&step); steps.push(step); } -pub(super) fn commit_full_rescan( - context: &SourceSetContext, - work_path: &Path, - recover_storage: bool, -) -> Result<(), AppError> { - match analyzer::rescan_and_commit_full(context, work_path) { - Ok(()) => Ok(()), - Err(_error) if recover_storage && storage_needs_recovery(context, work_path) => { - let storage_path = context.storage_path(work_path); - remove_storage_path(&storage_path).map_err(|remove_error| { - AppError::Runtime(format!( - "failed to remove corrupt storage '{}': {remove_error}", - storage_path.display() - )) - })?; - analyzer::rescan_and_commit_full(context, work_path) - .map_err(|retry_error| AppError::Runtime(retry_error.to_string())) +pub(super) fn attach_failed_receipt( + mut result: BuildResult, + source_set_name: &str, + receipt: &PlannedReceipt, +) -> BuildResult { + if let Some(step) = result + .steps + .iter_mut() + .rev() + .find(|step| step.source_set == source_set_name) + { + step.receipt = receipt.failed(); + } + result +} + +pub(super) fn receipt_after_success(plan: &StepPlan) -> SyncReceipt { + match plan { + StepPlan::Skip { receipt, .. } => receipt.clone(), + StepPlan::Execute { receipt, .. } => receipt.applied(), + } +} + +pub(super) fn attach_failed_plan_receipt( + result: BuildResult, + source_set_name: &str, + plan: &StepPlan, +) -> BuildResult { + match plan { + StepPlan::Execute { receipt, .. } => { + attach_failed_receipt(result, source_set_name, receipt) } - Err(error) => Err(AppError::Runtime(error.to_string())), + StepPlan::Skip { .. } => result, } } +pub(super) fn commit_full_observation( + context: &SourceSetContext, + prepared: &PreparedStateUpdate, +) -> Result<(), AppError> { + analyzer::commit_full_observation(context, prepared) + .map_err(|error| AppError::Runtime(error.to_string())) +} + pub(super) fn commit_step_state( source_set: &SourceSetConfig, context: &SourceSetContext, - work_path: &Path, commit: &StepCommit, ) -> Result<(), AppError> { match commit { @@ -442,27 +708,16 @@ pub(super) fn commit_step_state( source_set = source_set.name.as_str(), "committing prepared change-detection state" ); - analyzer::commit_success(context, work_path, prepared) + analyzer::commit_success(context, prepared) .map_err(|error| AppError::Runtime(error.to_string())) } - StepCommit::RescanFull { recover_storage } => { + StepCommit::FullObservation(prepared) => { debug!( source_set = source_set.name.as_str(), - recover_storage, "rescanning source-set state after full build" + "committing pre-build full source observation" ); - commit_full_rescan(context, work_path, *recover_storage) - } - } -} - -fn storage_needs_recovery(context: &SourceSetContext, work_path: &Path) -> bool { - match HashStorage::new(context.storage_path(work_path)).current_generation() { - Err(StorageError::Recoverable { .. }) => true, - Err(StorageError::Hard { reason, .. }) => { - let reason = reason.to_ascii_lowercase(); - reason.contains("invalid data") || reason.contains("corrupt") + commit_full_observation(context, prepared) } - Err(StorageError::ConcurrentStateModified { .. }) | Ok(_) => false, } } diff --git a/src/use_cases/check_syntax.rs b/src/use_cases/check_syntax.rs index 13ac47b..9574f12 100644 --- a/src/use_cases/check_syntax.rs +++ b/src/use_cases/check_syntax.rs @@ -437,7 +437,26 @@ fn run_edt_syntax( )); } - let inventory = SourceSetInventory::new(config); + let inventory = match SourceSetInventory::new(config) { + Ok(inventory) => inventory, + Err(error) => { + let error = AppError::from(error); + let error_message = error.to_string(); + return Err(SyntaxExecutionFailure::with_payload( + error, + failed_result( + "edt", + SyntaxCheckStatus::ToolFailed, + -1, + started, + vec![], + None, + Some(error_message), + None, + ), + )); + } + }; let source_sets = match resolve_edt_source_sets(&inventory, projects) { Ok(source_sets) => source_sets, Err(error) => { diff --git a/src/use_cases/dump_config.rs b/src/use_cases/dump_config.rs index dfec016..7a464ce 100644 --- a/src/use_cases/dump_config.rs +++ b/src/use_cases/dump_config.rs @@ -1,7 +1,10 @@ +use std::collections::{BTreeMap, BTreeSet}; use std::path::{Path, PathBuf}; use std::sync::Arc; use std::time::{Duration, Instant}; +use crate::change_detection::analyzer::{ManagedInventory, PreparedFileState, PreparedStateUpdate}; +use crate::change_detection::hash_storage::{HashStorage, ObservedHashStorage}; use crate::config::model::{AppConfig, BuilderBackend, SourceFormat, SourceSetPurpose}; use crate::domain::dump::{DumpMode, DumpResult, DumpSelectorResult}; use crate::domain::partial_dump_selector::PartialDumpSelector; @@ -9,6 +12,8 @@ use crate::domain::partial_dump_selector::PartialDumpSelector; use crate::domain::partial_dump_selector::{ PARTIAL_OBJECT_BLANK_ERROR, PARTIAL_OBJECT_CONTROL_ERROR, }; +use crate::domain::runtime_state::{BaselineRole, DumpTransactionId, StateGeneration}; +use crate::domain::source_set::SourceSetContext; use crate::platform::designer::DesignerDsl; use crate::platform::edt::EdtDsl; use crate::platform::edt_session::{EdtSessionHostOptions, EdtSessionManager}; @@ -48,7 +53,21 @@ use super::staged_publication::cleanup_staging_path; use super::staged_publication::{interruption_before_publish, StagedPublication}; #[cfg(test)] use crate::support::fs::metadata_sidecar_path; +use crate::use_cases::dump_execution::{EffectiveWriteScope, ShadowObservation}; +use crate::use_cases::dump_shadow::{ + inspect_baseline, managed_manifest, BaselineInspection, DumpShadow, EffectiveDumpMode, +}; +use crate::use_cases::runtime_state::{ + commit_dump_state_with_lock, lock_designer_state, recover_designer_state_with_lock, + DumpStateCommitRequest, +}; +use crate::use_cases::shadow_merge::{ + plan_manifest_merge, FileVersion, ManifestMergePlan, MergeAction, +}; use crate::use_cases::source_inventory::SourceSetInventory; +use crate::use_cases::source_publication::{ + recover_publication, ObservedStateGeneration, PublicationRequest, TargetIdentity, +}; #[cfg(test)] const DUMP_COMMAND: &str = crate::use_cases::context::CommandName::Dump.as_str(); @@ -88,6 +107,8 @@ struct ResolvedDumpTarget { platform_target_identity: String, lock_path: PathBuf, edt_base_project_name: Option, + configured_source_context: SourceSetContext, + platform_designer_context: SourceSetContext, } #[cfg(test)] @@ -828,7 +849,7 @@ fn cleanup_staging_on_interruption(staging_dir: &Path, error: AppError) -> AppEr } fn resolve_target(config: &AppConfig, args: &DumpArgs) -> Result { - let inventory = SourceSetInventory::new(config); + let inventory = SourceSetInventory::new(config).map_err(AppError::from)?; let (source_set, extension) = match (args.source_set.as_deref(), args.extension.as_deref()) { (Some(source_set_name), None) => { @@ -886,21 +907,28 @@ fn resolve_target(config: &AppConfig, args: &DumpArgs) -> Result inventory.designer_context(&source_set.name), + SourceFormat::Edt => inventory.edt_context(&source_set.name), + } + .cloned() + .ok_or_else(|| { + AppError::Runtime(format!( + "missing configured runtime context for source-set '{}'", + source_set.name + )) + })?; + let platform_designer_context = inventory + .designer_context(&source_set.name) + .cloned() + .ok_or_else(|| { + AppError::Runtime(format!( + "missing designer runtime context for source-set '{}'", + source_set.name + )) + })?; + let target_path = configured_source_context.path().to_path_buf(); + let platform_target_path = platform_designer_context.path().to_path_buf(); let canonical_target_path = nearest_existing_canonical_path(&target_path).map_err(|error| { AppError::Runtime(format!("failed to canonicalize target path: {error}")) })?; @@ -948,11 +976,14 @@ fn resolve_target(config: &AppConfig, args: &DumpArgs) -> Result SourceSetContext { + let identity = InfobaseIdentity::normalize(&crate::config::model::InfobaseConfig::file( + format!("File={}", root.join("ib").display()), + )) + .expect("identity"); + let layout = RuntimeStateLayout::new(root.join("runtime"), identity).expect("layout"); + let descriptor = RuntimeSourceDescriptor::new(RuntimeSourceIdentityInputs { + configured_source_identity: Path::new("source"), + source_root: root, + purpose: SourceSetPurpose::Configuration, + format: SourceFormat::Designer, + backend: BuilderBackend::Designer, + logical_role: LogicalSourceRole::DesignerSource, + }) + .expect("descriptor"); + SourceSetContext::new( + "main", + root.to_path_buf(), + layout.source_state("main", &descriptor), + ) + } + + fn bootstrap_dump(config: &AppConfig, source_set: &str, extension: Option<&str>) { + run_dump( + config, + &DumpArgs { + mode: DumpModeRequest::Full, + source_set: Some(source_set.to_owned()), + extension: extension.map(str::to_owned), + objects: vec![], + }, + ) + .expect("bootstrap dump"); + } + fn write_dump_script(path: &Path, calls_log: &Path, fail_pattern: Option<&str>, sleep_ms: u64) { let pattern_branch = fail_pattern .map(|pattern| { @@ -1025,7 +1096,7 @@ mod tests { format!("sleep {}", sleep_ms as f64 / 1000.0) }; let body = format!( - "args=\"$*\"\nout=\"\"\nprev=\"\"\nfor arg in \"$@\"; do\n if [ \"$prev\" = \"/Out\" ]; then out=\"$arg\"; fi\n prev=\"$arg\"\ndone\nif [ -n \"$out\" ]; then printf 'designer log for %s\\n' \"$args\" > \"$out\"; fi\nprintf '%s\\n' \"$args\" >> \"{}\"\n{}\n{}\nmkdir -p \"$(printf '%s' \"$args\" | awk '{{print $NF}}')\"\nexit 0", + "args=\"$*\"\nout=\"\"\ntarget=\"\"\nprev=\"\"\nfor arg in \"$@\"; do\n if [ \"$prev\" = \"/Out\" ]; then out=\"$arg\"; fi\n if [ \"$prev\" = \"/DumpConfigToFiles\" ]; then target=\"$arg\"; fi\n prev=\"$arg\"\ndone\nif [ -n \"$out\" ]; then printf 'designer log for %s\\n' \"$args\" > \"$out\"; fi\nprintf '%s\\n' \"$args\" >> \"{}\"\n{}\n{}\nmkdir -p \"$target\"\nprintf '\\n' > \"$target/ConfigDumpInfo.xml\"\nexit 0", calls_log.display(), sleep_branch, pattern_branch @@ -1053,7 +1124,7 @@ mod tests { format!("sleep {}", sleep_ms as f64 / 1000.0) }; let body = format!( - "args=\"$*\"\nprintf '%s\\n' \"$args\" >> \"{}\"\n{}\n{}\nmkdir -p \"$(printf '%s' \"$args\" | awk '{{print $NF}}')\"\nexit 0", + "args=\"$*\"\ntarget=\"$(printf '%s' \"$args\" | awk '{{print $NF}}')\"\nprintf '%s\\n' \"$args\" >> \"{}\"\n{}\n{}\nmkdir -p \"$target\"\nprintf '\\n' > \"$target/ConfigDumpInfo.xml\"\nexit 0", calls_log.display(), sleep_branch, pattern_branch @@ -1075,7 +1146,7 @@ mod tests { }) .unwrap_or_default(); let body = format!( - "args=\"$*\"\nout=\"\"\ntarget=\"\"\nextension_name=\"\"\nprev=\"\"\nfor arg in \"$@\"; do\n if [ \"$prev\" = \"/Out\" ]; then out=\"$arg\"; fi\n if [ \"$prev\" = \"/DumpConfigToFiles\" ]; then target=\"$arg\"; fi\n if [ \"$prev\" = \"-Extension\" ]; then extension_name=\"$arg\"; fi\n prev=\"$arg\"\ndone\nif [ -n \"$out\" ]; then printf 'designer log for %s\\n' \"$args\" > \"$out\"; fi\nprintf '%s\\n' \"$args\" >> \"{}\"\n{}\nmkdir -p \"$target\"\nif [ -n \"$extension_name\" ]; then\n config_xml='ExtensionProjectExtension'\nelse\n config_xml='BaseProject'\nfi\nprintf '%s\\n' \"$config_xml\" > \"$target/Configuration.xml\"\nif printf '%s' \"$args\" | grep -F -q -- '-partial'; then\n printf '\\n' > \"$target/PartialOnly.xml\"\nfi\nexit 0", + "args=\"$*\"\nout=\"\"\ntarget=\"\"\nextension_name=\"\"\nprev=\"\"\nfor arg in \"$@\"; do\n if [ \"$prev\" = \"/Out\" ]; then out=\"$arg\"; fi\n if [ \"$prev\" = \"/DumpConfigToFiles\" ]; then target=\"$arg\"; fi\n if [ \"$prev\" = \"-Extension\" ]; then extension_name=\"$arg\"; fi\n prev=\"$arg\"\ndone\nif [ -n \"$out\" ]; then printf 'designer log for %s\\n' \"$args\" > \"$out\"; fi\nprintf '%s\\n' \"$args\" >> \"{}\"\n{}\nmkdir -p \"$target\"\nif [ -n \"$extension_name\" ]; then\n config_xml='ExtensionProjectExtension'\nelse\n config_xml='BaseProject'\nfi\nprintf '%s\\n' \"$config_xml\" > \"$target/Configuration.xml\"\nprintf '\\n' > \"$target/ConfigDumpInfo.xml\"\nif printf '%s' \"$args\" | grep -F -q -- '-partial'; then\n printf '\\n' > \"$target/PartialOnly.xml\"\nfi\nexit 0", calls_log.display(), pattern_branch ); @@ -1092,7 +1163,7 @@ mod tests { }) .unwrap_or_default(); let body = format!( - "args=\"$*\"\ntarget=\"$(printf '%s' \"$args\" | awk '{{print $NF}}')\"\nextension_name=\"\"\nprev=\"\"\nfor arg in \"$@\"; do\n if [ \"$prev\" = \"-Extension\" ]; then extension_name=\"$arg\"; fi\n prev=\"$arg\"\ndone\nprintf '%s\\n' \"$args\" >> \"{}\"\n{}\nmkdir -p \"$target\"\nif [ -n \"$extension_name\" ]; then\n printf 'ExtensionProjectExtension\\n' > \"$target/Configuration.xml\"\nelse\n printf 'BaseProject\\n' > \"$target/Configuration.xml\"\nfi\nexit 0", + "args=\"$*\"\ntarget=\"$(printf '%s' \"$args\" | awk '{{print $NF}}')\"\nextension_name=\"\"\nprev=\"\"\nfor arg in \"$@\"; do\n if [ \"$prev\" = \"-Extension\" ]; then extension_name=\"$arg\"; fi\n prev=\"$arg\"\ndone\nprintf '%s\\n' \"$args\" >> \"{}\"\n{}\nmkdir -p \"$target\"\nif [ -n \"$extension_name\" ]; then\n printf 'ExtensionProjectExtension\\n' > \"$target/Configuration.xml\"\nelse\n printf 'BaseProject\\n' > \"$target/Configuration.xml\"\nfi\nprintf '\\n' > \"$target/ConfigDumpInfo.xml\"\nexit 0", calls_log.display(), pattern_branch ); @@ -1488,9 +1559,11 @@ exit 0"#, let dir = tempdir().expect("tempdir"); let script = dir.path().join("1cv8"); let calls = dir.path().join("calls.log"); - create_source_tree(dir.path()); - write_script(&script, &format!("touch '{}'", calls.display())); + fs::create_dir_all(dir.path().join("main")).expect("main"); + write_dump_script(&script, &calls, None, 0); let config = build_config(dir.path(), &dir.path().join("work"), &script); + bootstrap_dump(&config, "main", None); + fs::write(&calls, []).expect("clear calls"); let result = run_dump( &config, @@ -1682,6 +1755,8 @@ exit 0"#, platform_target_identity: "id".to_owned(), lock_path: dir.path().join(".lock"), edt_base_project_name: None, + configured_source_context: test_source_context(dir.path()), + platform_designer_context: test_source_context(dir.path()), }; let error = validate_publish_target(&resolved).expect_err("expected invalid"); @@ -1733,6 +1808,8 @@ exit 0"#, platform_target_identity: identity.clone(), lock_path: target.parent().expect("parent").join(".lock"), edt_base_project_name: None, + configured_source_context: test_source_context(dir.path()), + platform_designer_context: test_source_context(dir.path()), }; cleanup_orphan_dirs(&resolved).expect("cleanup"); @@ -1771,6 +1848,8 @@ exit 0"#, platform_target_identity: identity.clone(), lock_path: target.parent().expect("parent").join(".lock"), edt_base_project_name: None, + configured_source_context: test_source_context(dir.path()), + platform_designer_context: test_source_context(dir.path()), }; cleanup_orphan_dirs(&resolved).expect("cleanup"); @@ -1804,6 +1883,8 @@ exit 0"#, platform_target_identity: identity.clone(), lock_path: target.parent().expect("parent").join(".lock"), edt_base_project_name: None, + configured_source_context: test_source_context(dir.path()), + platform_designer_context: test_source_context(dir.path()), }; cleanup_orphan_dirs(&resolved).expect("cleanup"); @@ -1845,6 +1926,8 @@ exit 0"#, platform_target_identity: identity.clone(), lock_path: target.parent().expect("parent").join(".lock"), edt_base_project_name: None, + configured_source_context: test_source_context(dir.path()), + platform_designer_context: test_source_context(dir.path()), }; cleanup_orphan_dirs(&resolved).expect("cleanup"); @@ -1880,7 +1963,7 @@ exit 0"#, } #[test] - fn dump_incremental_creates_missing_target_dir() { + fn dump_incremental_without_baseline_runs_one_effective_full_shadow_dump() { let dir = tempdir().expect("tempdir"); let base = dir.path().join("base"); let work = dir.path().join("work"); @@ -1906,8 +1989,271 @@ exit 0"#, assert!(base.join("main").exists()); let calls = fs::read_to_string(calls).expect("calls"); assert!(calls.contains("/DumpConfigToFiles")); - assert!(calls.contains("-update")); - assert!(!calls.contains("-updateConfigDumpInfo")); + assert!(!calls.split_whitespace().any(|arg| arg == "-update")); + assert!(calls.contains("-updateConfigDumpInfo")); + } + + #[test] + fn retained_local_dump_path_remains_visible_to_next_build_analysis() { + let dir = tempdir().expect("tempdir"); + let base = dir.path().join("base"); + let work = dir.path().join("work"); + let script = dir.path().join("1cv8"); + let calls = dir.path().join("calls.log"); + create_source_tree(&base); + fs::remove_dir_all(base.join("main")).expect("remove target"); + write_script( + &script, + &format!( + "args=\"$*\"\ntarget=\"\"\nprev=\"\"\nfor arg in \"$@\"; do\n if [ \"$prev\" = \"/DumpConfigToFiles\" ]; then target=\"$arg\"; fi\n prev=\"$arg\"\ndone\nprintf '%s\\n' \"$args\" >> \"{}\"\nmkdir -p \"$target\"\nif ! printf ' %s ' \"$args\" | grep -q -- ' -update '; then printf 'baseline' > \"$target/Managed.txt\"; fi\nprintf '\\n' > \"$target/ConfigDumpInfo.xml\"\nexit 0", + calls.display() + ), + ); + let config = build_config(&base, &work, &script); + bootstrap_dump(&config, "main", None); + fs::write(base.join("main/Managed.txt"), "local").expect("local edit"); + + let result = run_dump( + &config, + &DumpArgs { + mode: DumpModeRequest::Incremental, + source_set: Some("main".to_owned()), + extension: None, + objects: vec![], + }, + ) + .expect("retained-local dump"); + + assert!(result.ok); + assert_eq!( + fs::read_to_string(base.join("main/Managed.txt")).expect("retained"), + "local" + ); + let context = resolve_target( + &config, + &DumpArgs { + mode: DumpModeRequest::Incremental, + source_set: Some("main".to_owned()), + extension: None, + objects: vec![], + }, + ) + .expect("resolved") + .configured_source_context; + let analysis = + crate::change_detection::analyzer::managed_inventory(&context).expect("next analysis"); + assert_eq!( + analysis + .requested + .iter() + .map(|change| change.rel_path.as_str()) + .collect::>(), + vec!["Managed.txt"] + ); + } + + #[test] + fn unchanged_private_shadow_rewrite_is_reported_as_processed_and_skipped() { + let dir = tempdir().expect("tempdir"); + let base = dir.path().join("base"); + let work = dir.path().join("work"); + let script = dir.path().join("1cv8"); + let calls = dir.path().join("calls.log"); + create_source_tree(&base); + fs::remove_dir_all(base.join("main")).expect("remove target"); + write_script( + &script, + &format!( + "args=\"$*\"\ntarget=\"\"\nprev=\"\"\nfor arg in \"$@\"; do if [ \"$prev\" = \"/DumpConfigToFiles\" ]; then target=\"$arg\"; fi; prev=\"$arg\"; done\nprintf '%s\\n' \"$args\" >> \"{}\"\nmkdir -p \"$target\"\nprintf 'stable' > \"$target/Stable.txt\"\nprintf '\\n' > \"$target/ConfigDumpInfo.xml\"\nexit 0", + calls.display() + ), + ); + let config = build_config(&base, &work, &script); + bootstrap_dump(&config, "main", None); + + let result = run_dump( + &config, + &DumpArgs { + mode: DumpModeRequest::Incremental, + source_set: Some("main".to_owned()), + extension: None, + objects: vec![], + }, + ) + .expect("incremental rewrite"); + + let receipt = serde_json::to_value(result.receipt).expect("receipt"); + assert_eq!(receipt["processed"][0]["path"], "Stable.txt"); + assert_eq!(receipt["skipped"][0]["path"], "Stable.txt"); + assert_eq!(receipt["processed"][0], receipt["skipped"][0]); + } + + #[test] + fn state_commit_failure_rolls_back_source_and_returns_exact_failed_receipt() { + let dir = tempdir().expect("tempdir"); + let base = dir.path().join("base"); + let work = dir.path().join("work"); + let script = dir.path().join("1cv8"); + let calls = dir.path().join("calls.log"); + create_source_tree(&base); + fs::remove_dir_all(base.join("main")).expect("remove target"); + write_script( + &script, + &format!( + "args=\"$*\"\ntarget=\"\"\nprev=\"\"\nfor arg in \"$@\"; do if [ \"$prev\" = \"/DumpConfigToFiles\" ]; then target=\"$arg\"; fi; prev=\"$arg\"; done\nprintf '%s\\n' \"$args\" >> \"{}\"\nmkdir -p \"$target\"\nprintf 'new' > \"$target/New.txt\"\nexit 0", + calls.display() + ), + ); + let config = build_config(&base, &work, &script); + + let failure = run_dump( + &config, + &DumpArgs { + mode: DumpModeRequest::Full, + source_set: Some("main".to_owned()), + extension: None, + objects: vec![], + }, + ) + .expect_err("missing private CDFI"); + + let receipt = + serde_json::to_value(&failure.payload.expect("payload").receipt).expect("receipt"); + assert_eq!(receipt["status"], "failed"); + assert_eq!(receipt["requested"][0]["path"], "New.txt"); + assert_eq!(receipt["processed"], serde_json::json!([])); + assert!(!base.join("main/New.txt").exists()); + } + + #[test] + fn cancellation_immediately_before_source_apply_keeps_source_and_state_unchanged() { + let dir = tempdir().expect("tempdir"); + let base = dir.path().join("base"); + let work = dir.path().join("work"); + let script = dir.path().join("1cv8"); + let calls = dir.path().join("calls.log"); + create_source_tree(&base); + fs::remove_dir_all(base.join("main")).expect("remove target"); + write_dump_script(&script, &calls, None, 0); + let config = build_config(&base, &work, &script); + bootstrap_dump(&config, "main", None); + write_script( + &script, + &format!( + "args=\"$*\"\ntarget=\"\"\nprev=\"\"\nfor arg in \"$@\"; do if [ \"$prev\" = \"/DumpConfigToFiles\" ]; then target=\"$arg\"; fi; prev=\"$arg\"; done\nprintf '%s\\n' \"$args\" >> \"{}\"\nmkdir -p \"$target\"\nprintf 'new' > \"$target/New.txt\"\nprintf '\\n' > \"$target/ConfigDumpInfo.xml\"\nexit 0", + calls.display() + ), + ); + let cancellation = CancellationToken::new(); + set_before_source_publication_hook({ + let cancellation = cancellation.clone(); + move || cancellation.cancel() + }); + let context = ExecutionContext::cli(crate::use_cases::context::CommandName::Dump) + .with_cancellation(cancellation); + + let failure = super::run_dump_with_context( + &context, + &config, + &DumpArgs { + mode: DumpModeRequest::Full, + source_set: Some("main".to_owned()), + extension: None, + objects: vec![], + }, + ) + .expect_err("cancelled before source apply"); + + let receipt = + serde_json::to_value(&failure.payload.expect("payload").receipt).expect("receipt"); + assert_eq!(receipt["status"], "failed"); + assert_eq!(receipt["requested"][0]["path"], "New.txt"); + assert_eq!(receipt["processed"], serde_json::json!([])); + assert!(!base.join("main/New.txt").exists()); + let resolved = resolve_target( + &config, + &DumpArgs { + mode: DumpModeRequest::Full, + source_set: Some("main".to_owned()), + extension: None, + objects: vec![], + }, + ) + .expect("resolved"); + assert_eq!( + crate::change_detection::hash_storage::HashStorage::new( + resolved.configured_source_context.storage_path() + ) + .current_generation() + .expect("generation"), + 1 + ); + } + + #[test] + fn restart_recovers_pending_source_publication_before_inventory() { + let dir = tempdir().expect("tempdir"); + let base = dir.path().join("base"); + let work = dir.path().join("work"); + let script = dir.path().join("1cv8"); + let calls = dir.path().join("calls.log"); + create_source_tree(&base); + fs::remove_dir_all(base.join("main")).expect("remove target"); + fs::create_dir_all(base.join("main")).expect("empty target"); + write_dump_script(&script, &calls, None, 0); + let config = build_config(&base, &work, &script); + let resolved = resolve_target( + &config, + &DumpArgs { + mode: DumpModeRequest::Full, + source_set: Some("main".to_owned()), + extension: None, + objects: vec![], + }, + ) + .expect("resolved"); + let proposed = dir.path().join("proposed"); + fs::create_dir(&proposed).expect("proposed"); + fs::write(proposed.join("Rogue.txt"), "pending").expect("rogue"); + let dump = + crate::use_cases::dump_shadow::managed_manifest(&proposed, &[]).expect("dump manifest"); + let plan = crate::use_cases::shadow_merge::plan_manifest_merge( + &std::collections::BTreeMap::new(), + &std::collections::BTreeMap::new(), + &dump, + ); + let transaction = resolved + .configured_source_context + .transactions_dir() + .join("source-publication"); + fs::create_dir_all(transaction.parent().expect("transactions")) + .expect("transactions directory"); + let pending = crate::use_cases::source_publication::PublicationRequest::builder( + resolved.configured_source_context.path(), + &proposed, + &plan, + ) + .transaction_root(&transaction) + .generation(1) + .target_identity(crate::use_cases::source_publication::TargetIdentity::new( + resolved.target_identity.clone(), + )) + .dump_transaction_id(crate::domain::runtime_state::DumpTransactionId::new()) + .prepare() + .expect("prepare pending") + .apply() + .expect("apply pending"); + drop(pending); + assert!(base.join("main/Rogue.txt").exists()); + + bootstrap_dump(&config, "main", None); + + assert!(!base.join("main/Rogue.txt").exists()); + let analysis = crate::change_detection::analyzer::managed_inventory( + &resolved.configured_source_context, + ) + .expect("post-recovery inventory"); + assert!(analysis.requested.is_empty()); } #[test] @@ -1920,6 +2266,9 @@ exit 0"#, create_source_tree(&base); write_dump_script(&script, &calls, None, 0); let config = build_config(&base, &work, &script); + fs::remove_dir_all(base.join("ext")).expect("remove extension"); + bootstrap_dump(&config, "ext", Some("ext")); + fs::write(&calls, "").expect("clear calls"); let result = run_dump( &config, @@ -1935,8 +2284,8 @@ exit 0"#, assert!(result.ok); let calls = fs::read_to_string(calls).expect("calls"); assert!(calls.contains("/DumpConfigToFiles")); - assert!(calls.contains("-update")); - assert!(!calls.contains("-updateConfigDumpInfo")); + assert!(calls.split_whitespace().any(|arg| arg == "-update")); + assert!(calls.contains("-updateConfigDumpInfo")); assert!(calls.contains("-Extension")); assert!(calls.contains("ext")); } @@ -2012,13 +2361,15 @@ exit 0"#, write_script( &script, &format!( - "args=\"$*\"\nout=\"\"\nlist=\"\"\nprev=\"\"\nfor arg in \"$@\"; do\n if [ \"$prev\" = \"/Out\" ]; then out=\"$arg\"; fi\n if [ \"$prev\" = \"-listFile\" ]; then list=\"$arg\"; fi\n prev=\"$arg\"\ndone\nif [ -n \"$out\" ]; then printf 'designer log for %s\\n' \"$args\" > \"$out\"; fi\nif [ -n \"$list\" ]; then cp \"$list\" \"{}\"; fi\nprintf '%s\\n' \"$args\" >> \"{}\"\nexit 0", + "args=\"$*\"\nout=\"\"\nlist=\"\"\ntarget=\"\"\nprev=\"\"\nfor arg in \"$@\"; do\n if [ \"$prev\" = \"/Out\" ]; then out=\"$arg\"; fi\n if [ \"$prev\" = \"-listFile\" ]; then list=\"$arg\"; fi\n if [ \"$prev\" = \"/DumpConfigToFiles\" ]; then target=\"$arg\"; fi\n prev=\"$arg\"\ndone\nif [ -n \"$out\" ]; then printf 'designer log for %s\\n' \"$args\" > \"$out\"; fi\nif [ -n \"$list\" ]; then cp \"$list\" \"{}\"; fi\nprintf '%s\\n' \"$args\" >> \"{}\"\nmkdir -p \"$target\"\nprintf '\\n' > \"$target/ConfigDumpInfo.xml\"\nexit 0", captured_list.display(), calls.display(), ), ); let config = build_config(&base, &work, &script); fs::remove_dir_all(base.join("main")).expect("remove target"); + bootstrap_dump(&config, "main", None); + fs::write(&calls, "").expect("clear calls"); let result = run_dump( &config, @@ -2052,7 +2403,7 @@ exit 0"#, assert!(calls.contains("/DumpConfigToFiles")); assert!(calls.contains("-partial")); assert!(calls.contains("-listFile")); - assert!(!calls.contains("-updateConfigDumpInfo")); + assert!(calls.contains("-updateConfigDumpInfo")); assert_eq!( fs::read_to_string(captured_list).expect("captured list"), "Catalog.Items\nDocument.Order\n" @@ -2070,6 +2421,9 @@ exit 0"#, create_source_tree(&base); write_dump_script(&script, &calls, None, 0); let config = build_config(&base, &work, &script); + fs::remove_dir_all(base.join("ext")).expect("remove extension"); + bootstrap_dump(&config, "ext", Some("ext")); + fs::write(&calls, "").expect("clear calls"); let result = run_dump( &config, @@ -2099,6 +2453,9 @@ exit 0"#, create_source_tree(&base); write_dump_script(&script, &calls, Some("-partial"), 0); let config = build_config(&base, &work, &script); + fs::remove_dir_all(base.join("main")).expect("remove target"); + bootstrap_dump(&config, "main", None); + fs::write(&calls, "").expect("clear calls"); let failure = run_dump( &config, @@ -2126,6 +2483,9 @@ exit 0"#, create_source_tree(&base); write_ibcmd_dump_script(&script, &calls, None, 0); let config = build_config_with_builder(&base, &work, &script, BuilderBackend::Ibcmd); + fs::remove_dir_all(base.join("main")).expect("remove target"); + bootstrap_dump(&config, "main", None); + fs::write(&calls, "").expect("clear calls"); let result = run_dump( &config, @@ -2156,9 +2516,11 @@ exit 0"#, let work = dir.path().join("work"); let script = dir.path().join("ibcmd"); let calls = dir.path().join("calls.log"); - create_source_tree(&base); + fs::create_dir_all(base.join("main")).expect("main"); write_ibcmd_dump_script(&script, &calls, None, 0); let config = build_config_with_builder(&base, &work, &script, BuilderBackend::Ibcmd); + bootstrap_dump(&config, "main", None); + fs::write(&calls, []).expect("clear calls"); let result = run_dump( &config, @@ -2201,6 +2563,9 @@ exit 0"#, create_source_tree(&base); write_ibcmd_dump_script(&script, &calls, None, 0); let config = build_config_with_builder(&base, &work, &script, BuilderBackend::Ibcmd); + fs::remove_dir_all(base.join("ext")).expect("remove extension"); + bootstrap_dump(&config, "ext", Some("ext")); + fs::write(&calls, "").expect("clear calls"); let result = run_dump( &config, @@ -2235,6 +2600,9 @@ exit 0"#, create_source_tree(&base); write_ibcmd_dump_script(&script, &calls, Some("--sync"), 0); let config = build_config_with_builder(&base, &work, &script, BuilderBackend::Ibcmd); + fs::remove_dir_all(base.join("main")).expect("remove target"); + bootstrap_dump(&config, "main", None); + fs::write(&calls, "").expect("clear calls"); let failure = run_dump( &config, @@ -2285,6 +2653,11 @@ exit 0"#, .expect_err("failure"); assert_eq!(failure.error.kind(), UseCaseErrorKind::Platform); + let receipt = + serde_json::to_value(&failure.payload.expect("payload").receipt).expect("receipt"); + assert_eq!(receipt["status"], "failed"); + assert_eq!(receipt["requested"], serde_json::json!([])); + assert_eq!(receipt["processed"], serde_json::json!([])); assert_eq!( fs::read_to_string(base.join("main").join("old.txt")).expect("old"), "keep me" @@ -2292,7 +2665,7 @@ exit 0"#, } #[test] - fn dump_full_success_replaces_old_target() { + fn dump_full_without_baseline_conflicts_and_preserves_old_target() { let dir = tempdir().expect("tempdir"); let base = dir.path().join("base"); let work = dir.path().join("work"); @@ -2303,7 +2676,7 @@ exit 0"#, let config = build_config(&base, &work, &script); fs::write(base.join("main").join("old.txt"), "old").expect("old"); - let result = run_dump( + let failure = run_dump( &config, &DumpArgs { mode: DumpModeRequest::Full, @@ -2312,14 +2685,18 @@ exit 0"#, objects: vec![], }, ) - .expect("dump"); + .expect_err("conflict"); - assert!(result.ok); - assert!(!base.join("main").join("old.txt").exists()); + assert_eq!( + serde_json::to_value(&failure.payload.expect("payload").receipt).expect("receipt") + ["status"], + "conflict" + ); + assert!(base.join("main").join("old.txt").exists()); } #[test] - fn ibcmd_dump_full_uses_staging_dir_and_atomic_publish() { + fn ibcmd_dump_full_uses_private_staging_and_preserves_conflicted_source() { let dir = tempdir().expect("tempdir"); let base = dir.path().join("base"); let work = dir.path().join("work"); @@ -2330,7 +2707,7 @@ exit 0"#, let config = build_config_with_builder(&base, &work, &script, BuilderBackend::Ibcmd); fs::write(base.join("main").join("old.txt"), "old").expect("old"); - let result = run_dump( + let failure = run_dump( &config, &DumpArgs { mode: DumpModeRequest::Full, @@ -2339,13 +2716,17 @@ exit 0"#, objects: vec![], }, ) - .expect("dump"); + .expect_err("conflict"); let calls = fs::read_to_string(calls).expect("calls"); - assert!(result.ok); + assert_eq!( + serde_json::to_value(&failure.payload.expect("payload").receipt).expect("receipt") + ["status"], + "conflict" + ); assert!(calls.contains("--force")); assert!(calls.contains(".dump-stage-")); - assert!(!base.join("main").join("old.txt").exists()); + assert!(base.join("main").join("old.txt").exists()); } #[test] @@ -2365,7 +2746,7 @@ exit 0"#, ) .with_credentials(Some("Admin".to_owned()), Some("secret".to_owned())); - let result = run_dump( + let failure = run_dump( &config, &DumpArgs { mode: DumpModeRequest::Full, @@ -2374,9 +2755,13 @@ exit 0"#, objects: vec![], }, ) - .expect("dump"); + .expect_err("bootstrap conflict"); - assert!(result.ok); + assert_eq!( + serde_json::to_value(&failure.payload.expect("payload").receipt).expect("receipt") + ["status"], + "conflict" + ); let calls = fs::read_to_string(calls).expect("calls"); assert!(calls.contains("--dbms PostgreSQL")); assert!(calls.contains("--database-server localhost")); @@ -2418,7 +2803,7 @@ exit 0"#, } #[test] - fn ibcmd_dump_incremental_uses_sync_against_resolved_target() { + fn ibcmd_dump_incremental_uses_sync_against_private_shadow() { let dir = tempdir().expect("tempdir"); let base = dir.path().join("base"); let work = dir.path().join("work"); @@ -2428,6 +2813,8 @@ exit 0"#, write_ibcmd_dump_script(&script, &calls, None, 0); let config = build_config_with_builder(&base, &work, &script, BuilderBackend::Ibcmd); fs::remove_dir_all(base.join("main")).expect("remove target"); + bootstrap_dump(&config, "main", None); + fs::write(&calls, "").expect("clear calls"); let result = run_dump( &config, @@ -2443,11 +2830,12 @@ exit 0"#, assert!(result.ok); let calls = fs::read_to_string(calls).expect("calls"); assert!(calls.contains("--sync")); - assert!(calls.contains(base.join("main").display().to_string().as_str())); + assert!(!calls.contains(base.join("main").display().to_string().as_str())); + assert!(calls.contains("dump-shadow-")); } #[test] - fn dump_full_edt_designer_updates_designer_mirror_and_publishes_edt_target() { + fn dump_full_edt_designer_bootstrap_conflict_preserves_source_and_unmanaged_entries() { let dir = tempdir().expect("tempdir"); let base = dir.path().join("base"); let work = dir.path().join("work"); @@ -2460,8 +2848,10 @@ exit 0"#, write_edt_import_script(&edt, &edt_calls); let config = build_edt_config(&base, &work, &designer, &edt, BuilderBackend::Designer); fs::write(base.join("main").join("stale.txt"), "stale").expect("stale"); + fs::create_dir_all(base.join("main/.git")).expect("git metadata"); + fs::write(base.join("main/.git/keep"), "unmanaged").expect("unmanaged marker"); - let result = run_dump( + let failure = run_dump( &config, &DumpArgs { mode: DumpModeRequest::Full, @@ -2470,13 +2860,21 @@ exit 0"#, objects: vec![], }, ) - .expect("dump"); + .expect_err("bootstrap conflict"); + let result = failure.payload.expect("conflict payload"); - assert!(result.ok); - assert_eq!(result.target_path, base.join("main")); + assert!(!result.ok); + assert_eq!( + serde_json::to_value(&result.receipt).expect("receipt")["status"], + "conflict" + ); assert_native_edt_project(&base.join("main")); - assert!(!base.join("main").join("stale.txt").exists()); - assert!(work + assert!(base.join("main").join("stale.txt").exists()); + assert_eq!( + fs::read_to_string(base.join("main/.git/keep")).expect("unmanaged marker"), + "unmanaged" + ); + assert!(!work .join("designer") .join("main") .join("Configuration.xml") @@ -2484,13 +2882,13 @@ exit 0"#, let designer_calls = fs::read_to_string(designer_calls).expect("designer calls"); let edt_calls = fs::read_to_string(edt_calls).expect("edt calls"); - assert!(designer_calls.contains(work.join("designer").display().to_string().as_str())); - assert!(edt_calls.contains(work.join("designer/main").display().to_string().as_str())); + assert!(!designer_calls.contains(work.join("designer/main").display().to_string().as_str())); + assert!(!edt_calls.contains(work.join("designer/main").display().to_string().as_str())); assert!(edt_calls.contains(work.join("edt-workspace").display().to_string().as_str())); } #[test] - fn dump_partial_edt_designer_bootstraps_missing_or_invalid_designer_snapshot() { + fn dump_partial_edt_designer_bootstrap_runs_one_effective_full_shadow_dump() { let dir = tempdir().expect("tempdir"); let base = dir.path().join("base"); let work = dir.path().join("work"); @@ -2522,26 +2920,21 @@ exit 0"#, assert!(result.ok); assert_native_edt_project(&base.join("main")); - assert!(work + assert!(!work .join("designer") .join("main") .join("Configuration.xml") .exists()); - assert!(work - .join("designer") - .join("main") - .join("PartialOnly.xml") - .exists()); - + assert!(work.join("designer/main/BrokenMirror.xml").exists()); let designer_calls = fs::read_to_string(designer_calls).expect("designer calls"); let edt_calls = fs::read_to_string(edt_calls).expect("edt calls"); - assert_eq!(designer_calls.matches("/DumpConfigToFiles").count(), 2); - assert!(designer_calls.contains("-partial")); + assert_eq!(designer_calls.matches("/DumpConfigToFiles").count(), 1); + assert!(!designer_calls.contains("-partial")); assert_eq!(edt_calls.matches("-command import").count(), 1); } #[test] - fn dump_incremental_edt_designer_bootstrap_is_full_then_follow_up_uses_update() { + fn dump_incremental_edt_designer_bootstrap_runs_one_effective_full_shadow_dump() { let dir = tempdir().expect("tempdir"); let base = dir.path().join("base"); let work = dir.path().join("work"); @@ -2572,16 +2965,48 @@ exit 0"#, .lines() .filter(|line| line.contains("/DumpConfigToFiles")) .collect::>(); - assert_eq!(dump_calls.len(), 2); - assert!(!dump_calls[0].contains("-update")); - assert!(!dump_calls[0].contains("-updateConfigDumpInfo")); - assert!(dump_calls[1].contains("-update")); - assert!(!dump_calls[1].contains("-updateConfigDumpInfo")); + assert_eq!(dump_calls.len(), 1); + assert!(!dump_calls[0].split_whitespace().any(|arg| arg == "-update")); + assert!(dump_calls[0].contains("-updateConfigDumpInfo")); let edt_calls = fs::read_to_string(edt_calls).expect("edt calls"); assert_eq!(edt_calls.matches("-command import").count(), 1); } + #[test] + fn edt_incremental_promotes_to_full_when_configured_baseline_is_corrupt() { + let dir = tempdir().expect("tempdir"); + let base = dir.path().join("base"); + let work = dir.path().join("work"); + let designer = dir.path().join("1cv8"); + let edt = dir.path().join("edt/1cedtcli"); + let designer_calls = dir.path().join("designer-calls.log"); + let edt_calls = dir.path().join("edt-calls.log"); + create_edt_source_tree(&base); + write_designer_dump_script_for_edt(&designer, &designer_calls, None); + write_edt_import_script(&edt, &edt_calls); + let config = build_edt_config(&base, &work, &designer, &edt, BuilderBackend::Designer); + let args = DumpArgs { + mode: DumpModeRequest::Incremental, + source_set: Some("main".to_owned()), + extension: None, + objects: vec![], + }; + run_dump(&config, &args).expect("bootstrap"); + let resolved = resolve_target(&config, &args).expect("resolved"); + let configured = resolved + .configured_source_context + .baseline(BaselineRole::ConfiguredSource, StateGeneration::new(1)); + fs::write(configured.path().join("manifest.json"), b"corrupt").expect("corrupt baseline"); + fs::write(&designer_calls, "").expect("clear calls"); + + run_dump(&config, &args).expect("promoted full"); + + let calls = fs::read_to_string(designer_calls).expect("calls"); + assert_eq!(calls.matches("/DumpConfigToFiles").count(), 1); + assert!(!calls.split_whitespace().any(|arg| arg == "-update")); + } + #[test] fn dump_full_edt_extension_infers_base_project_name_from_configuration_source_set() { let dir = tempdir().expect("tempdir"); @@ -2643,7 +3068,7 @@ exit 0"#, assert!(result.ok); assert!(base.join("main").join(".project").exists()); - assert!(work + assert!(!work .join("designer") .join("main") .join("Configuration.xml") @@ -2652,8 +3077,8 @@ exit 0"#, let ibcmd_calls = fs::read_to_string(ibcmd_calls).expect("ibcmd calls"); let edt_calls = fs::read_to_string(edt_calls).expect("edt calls"); assert!(ibcmd_calls.contains("--force")); - assert!(ibcmd_calls.contains(work.join("designer").display().to_string().as_str())); - assert!(edt_calls.contains(work.join("designer/main").display().to_string().as_str())); + assert!(!ibcmd_calls.contains(work.join("designer/main").display().to_string().as_str())); + assert!(!edt_calls.contains(work.join("designer/main").display().to_string().as_str())); } #[test] @@ -3191,6 +3616,7 @@ exit 0"#, platform_log_path: Some(PathBuf::from("/tmp/platform.log")), duration_ms: 5, message: Some("ok".to_owned()), + receipt: Default::default(), }; let json = serde_json::to_value(result).expect("json"); diff --git a/src/use_cases/dump_config/coordinator.rs b/src/use_cases/dump_config/coordinator.rs index 0f2af7b..50838e4 100644 --- a/src/use_cases/dump_config/coordinator.rs +++ b/src/use_cases/dump_config/coordinator.rs @@ -1,5 +1,197 @@ use super::*; +#[cfg(test)] +thread_local! { + static BEFORE_SOURCE_PUBLICATION: std::cell::RefCell>> = + const { std::cell::RefCell::new(None) }; +} + +#[cfg(test)] +pub(super) fn set_before_source_publication_hook(hook: impl FnOnce() + 'static) { + BEFORE_SOURCE_PUBLICATION.with(|slot| slot.replace(Some(Box::new(hook)))); +} + +#[cfg(test)] +fn run_before_source_publication_hook() { + BEFORE_SOURCE_PUBLICATION.with(|slot| { + if let Some(hook) = slot.take() { + hook(); + } + }); +} + +#[cfg(not(test))] +fn run_before_source_publication_hook() {} + +enum PrivateDumpOutcome { + Applied { + platform_result: PlatformCommandResult, + cleanup_message: Option, + receipt: crate::domain::sync_receipt::SyncReceipt, + }, + Conflict(crate::domain::sync_receipt::SyncReceipt), +} + +fn effective_dump_mode(mode: EffectiveDumpMode) -> DumpMode { + match mode { + EffectiveDumpMode::Full => DumpMode::Full, + EffectiveDumpMode::Incremental => DumpMode::Incremental, + EffectiveDumpMode::Partial => DumpMode::Partial, + } +} + +fn observed_generation(context: &SourceSetContext) -> Result { + match HashStorage::new(context.storage_path()) + .observe_state() + .map_err(|error| AppError::Runtime(error.to_string()))? + { + ObservedHashStorage::MissingPath | ObservedHashStorage::ExistingUninitialized(_) => Ok(0), + ObservedHashStorage::Initialized(snapshot) => Ok(snapshot.generation), + ObservedHashStorage::Recoverable(observation) => Ok(observation.generation()), + } +} + +fn current_baseline_manifest( + context: &SourceSetContext, + generation: StateGeneration, +) -> Result, AppError> { + match inspect_baseline(&context.baseline(BaselineRole::ConfiguredSource, generation)) + .map_err(|error| AppError::Runtime(error.to_string()))? + { + BaselineInspection::Valid(baseline) => Ok(baseline + .files() + .iter() + .map(|file| (file.path().to_owned(), file.sha256())) + .collect()), + BaselineInspection::Missing | BaselineInspection::Corrupt(_) => Ok(BTreeMap::new()), + } +} + +fn baseline_is_valid( + context: &SourceSetContext, + role: BaselineRole, + generation: StateGeneration, +) -> Result { + Ok(matches!( + inspect_baseline(&context.baseline(role, generation)) + .map_err(|error| AppError::Runtime(error.to_string()))?, + BaselineInspection::Valid(_) + )) +} + +fn resolved_for_shadows( + resolved: &ResolvedDumpTarget, + configured_shadow: &Path, + platform_shadow: &Path, +) -> Result { + if resolved.platform_target_path != resolved.platform_designer_context.path() { + return Err(AppError::Runtime( + "resolved platform target does not match its Designer context".to_owned(), + )); + } + let mut shadow = resolved.clone(); + shadow.target_path = configured_shadow.to_path_buf(); + shadow.canonical_target_path = nearest_existing_canonical_path(configured_shadow) + .map_err(|error| AppError::Runtime(format!("failed to resolve dump shadow: {error}")))?; + shadow.target_identity = stable_path_identity(&shadow.canonical_target_path); + shadow.platform_target_path = platform_shadow.to_path_buf(); + shadow.canonical_platform_target_path = nearest_existing_canonical_path(platform_shadow) + .map_err(|error| { + AppError::Runtime(format!("failed to resolve platform shadow: {error}")) + })?; + shadow.platform_target_identity = stable_path_identity(&shadow.canonical_platform_target_path); + Ok(shadow) +} + +fn prepare_dump_observation( + observed: &PreparedStateUpdate, + post_publication: &ManagedInventory, + merge: &ManifestMergePlan, +) -> Result { + let current = post_publication + .current + .iter() + .map(|file| (file.rel_path.as_str(), file)) + .collect::>(); + let retain_local = merge + .entries() + .iter() + .any(|entry| entry.action() == MergeAction::RetainLocal); + let mut snapshot = Vec::new(); + for entry in merge.entries() { + let version = match entry.action() { + MergeAction::Apply | MergeAction::Converged | MergeAction::NoOp => entry.dump(), + MergeAction::RetainLocal => entry.baseline(), + MergeAction::Conflict => { + return Err(AppError::Runtime( + "conflicted merge cannot prepare dump observation".to_owned(), + )) + } + }; + let FileVersion::Present(hash) = version else { + continue; + }; + let hash = hash + .iter() + .map(|byte| format!("{byte:02x}")) + .collect::(); + let mtime_ns = current + .get(entry.path()) + .filter(|file| file.hash == hash) + .map_or(0, |file| file.mtime_ns); + snapshot.push(PreparedFileState { + rel_path: entry.path().to_owned(), + mtime_ns, + hash, + }); + } + Ok(PreparedStateUpdate { + snapshot, + scan_started_at: if retain_local { + 0 + } else { + post_publication.prepared.scan_started_at + }, + observed_storage: observed.observed_storage.clone(), + }) +} + +fn rollback_publication_failure( + failure: AppError, + transaction_root: &Path, + context: &SourceSetContext, + identity: &TargetIdentity, + observed_generation: u64, +) -> AppError { + match recover_publication( + transaction_root, + context.path(), + identity, + ObservedStateGeneration::new(observed_generation), + ) { + Ok(()) => failure, + Err(recovery_error) => AppError::Runtime(format!( + "dump operation failed ({failure}); source rollback also failed ({recovery_error})" + )), + } +} + +fn observed_publication_state( + context: &SourceSetContext, + generation: u64, +) -> Result { + match HashStorage::new(context.storage_path()) + .current_dump_transaction_id() + .map_err(|error| AppError::Runtime(error.to_string()))? + { + Some(transaction_id) => Ok(ObservedStateGeneration::with_dump_transaction( + generation, + transaction_id, + )), + None => Ok(ObservedStateGeneration::new(generation)), + } +} + pub(super) fn run_dump_with_context( context: &ExecutionContext, config: &AppConfig, @@ -219,162 +411,401 @@ pub(super) fn run_dump_with_context( } } - let partial_objects = partial_objects.as_deref(); - let edt_binary = edt_binary.as_deref(); - let result = match ( - config.format, - &mode, - &config.builder, - partial_objects, - edt_binary, - ) { - (SourceFormat::Designer, DumpMode::Incremental, BuilderBackend::Designer, _, _) => { - run_incremental_dump_designer( - context, - config, - &resolved, - location.path.as_path(), - utilities.runner_for(UtilityType::V8), - ) + let mut failure_receipt = crate::domain::sync_receipt::SyncReceipt::empty_failed(); + let result = (|| -> Result { + let state_lock = lock_designer_state(&resolved.configured_source_context) + .map_err(|error| AppError::Runtime(error.to_string()))?; + recover_designer_state_with_lock(&resolved.configured_source_context, &state_lock) + .map_err(|error| AppError::Runtime(error.to_string()))?; + ensure_dir(resolved.configured_source_context.path()).map_err(|error| { + AppError::Runtime(format!("failed to create configured source root: {error}")) + })?; + let observed_generation = observed_generation(&resolved.configured_source_context)?; + let generation = StateGeneration::new(observed_generation); + let publication_root = resolved + .configured_source_context + .transactions_dir() + .join("source-publication"); + let publication_identity = TargetIdentity::new(resolved.target_identity.clone()); + recover_publication( + &publication_root, + resolved.configured_source_context.path(), + &publication_identity, + observed_publication_state(&resolved.configured_source_context, observed_generation)?, + ) + .map_err(|error| AppError::Runtime(error.to_string()))?; + let inventory = crate::change_detection::analyzer::managed_inventory( + &resolved.configured_source_context, + ) + .map_err(|error| AppError::Runtime(error.to_string()))?; + if inventory.prepared.observed_storage.generation() != observed_generation { + return Err(AppError::Runtime( + "runtime-state generation changed during dump recovery".to_owned(), + )); } - (SourceFormat::Designer, DumpMode::Incremental, BuilderBackend::Ibcmd, _, _) => { - run_incremental_dump_ibcmd( - context, - config, - &resolved, - location.path.as_path(), - utilities.runner_for(UtilityType::Ibcmd), + + let baseline = current_baseline_manifest(&resolved.configured_source_context, generation)?; + let source = managed_manifest( + resolved.configured_source_context.path(), + resolved.configured_source_context.excluded_roots(), + ) + .map_err(|error| AppError::Runtime(error.to_string()))?; + let shadow_request = if config.format == SourceFormat::Edt + && !matches!(args.mode, DumpModeRequest::Full) + && !baseline_is_valid( + &resolved.configured_source_context, + BaselineRole::ConfiguredSource, + generation, + )? { + DumpModeRequest::Full + } else { + args.mode + }; + let platform_shadow = DumpShadow::prepare( + &resolved.configured_source_context, + if config.format == SourceFormat::Edt { + BaselineRole::EdtPlatformDesigner + } else { + BaselineRole::ConfiguredSource + }, + generation, + shadow_request, + ) + .map_err(|error| AppError::Runtime(error.to_string()))?; + let configured_shadow = if config.format == SourceFormat::Edt { + Some( + DumpShadow::prepare( + &resolved.configured_source_context, + BaselineRole::ConfiguredSource, + generation, + DumpModeRequest::Full, + ) + .map_err(|error| AppError::Runtime(error.to_string()))?, ) - } - (SourceFormat::Designer, DumpMode::Full, BuilderBackend::Designer, _, _) => { - run_full_dump_designer( + } else { + None + }; + let configured_shadow_root = configured_shadow + .as_ref() + .map_or(platform_shadow.path(), DumpShadow::path); + let shadow_resolved = + resolved_for_shadows(&resolved, configured_shadow_root, platform_shadow.path())?; + let effective_mode = effective_dump_mode(platform_shadow.mode()); + let partial_objects = if platform_shadow.mode() == EffectiveDumpMode::Full { + None + } else { + partial_objects.as_deref() + }; + let baseline_paths = baseline.keys().cloned().collect::>(); + let write_scope = if platform_shadow.mode() == EffectiveDumpMode::Full { + EffectiveWriteScope::Full { + baseline: &baseline_paths, + } + } else { + EffectiveWriteScope::Incremental + }; + let shadow_observation = + ShadowObservation::normalize_and_capture(configured_shadow_root, &[]) + .map_err(|error| AppError::Runtime(error.to_string()))?; + let edt_binary = edt_binary.as_deref(); + let command_result = match ( + config.format, + &effective_mode, + &config.builder, + partial_objects, + edt_binary, + ) { + (SourceFormat::Designer, DumpMode::Incremental, BuilderBackend::Designer, _, _) => { + run_incremental_dump_designer( + context, + config, + &shadow_resolved, + location.path.as_path(), + utilities.runner_for(UtilityType::V8), + ) + } + (SourceFormat::Designer, DumpMode::Incremental, BuilderBackend::Ibcmd, _, _) => { + run_incremental_dump_ibcmd( + context, + config, + &shadow_resolved, + location.path.as_path(), + utilities.runner_for(UtilityType::Ibcmd), + ) + } + (SourceFormat::Designer, DumpMode::Full, BuilderBackend::Designer, _, _) => { + run_full_dump_designer( + context, + config, + &shadow_resolved, + location.path.as_path(), + utilities.runner_for(UtilityType::V8), + ) + } + (SourceFormat::Designer, DumpMode::Full, BuilderBackend::Ibcmd, _, _) => { + run_full_dump_ibcmd( + context, + config, + &shadow_resolved, + location.path.as_path(), + utilities.runner_for(UtilityType::Ibcmd), + ) + } + ( + SourceFormat::Designer, + DumpMode::Partial, + BuilderBackend::Designer, + Some(objects), + _, + ) => run_partial_dump_designer( context, config, - &resolved, + &shadow_resolved, location.path.as_path(), utilities.runner_for(UtilityType::V8), - ) - } - (SourceFormat::Designer, DumpMode::Full, BuilderBackend::Ibcmd, _, _) => { - run_full_dump_ibcmd( + objects, + ), + ( + SourceFormat::Designer, + DumpMode::Partial, + BuilderBackend::Ibcmd, + Some(objects), + _, + ) => run_partial_dump_ibcmd( context, config, - &resolved, + &shadow_resolved, location.path.as_path(), utilities.runner_for(UtilityType::Ibcmd), - ) - } - (SourceFormat::Designer, DumpMode::Partial, BuilderBackend::Designer, Some(objects), _) => { - run_partial_dump_designer( - context, - config, - &resolved, - location.path.as_path(), - utilities.runner_for(UtilityType::V8), objects, - ) - } - (SourceFormat::Designer, DumpMode::Partial, BuilderBackend::Ibcmd, Some(objects), _) => { - run_partial_dump_ibcmd( + ), + ( + SourceFormat::Edt, + DumpMode::Incremental, + BuilderBackend::Designer, + _, + Some(edt_binary), + ) => run_incremental_dump_edt_designer( context, config, - &resolved, + &shadow_resolved, location.path.as_path(), - utilities.runner_for(UtilityType::Ibcmd), - objects, - ) - } - ( - SourceFormat::Edt, - DumpMode::Incremental, - BuilderBackend::Designer, - _, - Some(edt_binary), - ) => run_incremental_dump_edt_designer( - context, - config, - &resolved, - location.path.as_path(), - edt_binary, - utilities.runner_for(UtilityType::V8), - utilities.runner_for(UtilityType::EdtCli), - ), - (SourceFormat::Edt, DumpMode::Incremental, BuilderBackend::Ibcmd, _, Some(edt_binary)) => { - run_incremental_dump_edt_ibcmd( + edt_binary, + utilities.runner_for(UtilityType::V8), + utilities.runner_for(UtilityType::EdtCli), + ), + ( + SourceFormat::Edt, + DumpMode::Incremental, + BuilderBackend::Ibcmd, + _, + Some(edt_binary), + ) => run_incremental_dump_edt_ibcmd( context, config, - &resolved, + &shadow_resolved, location.path.as_path(), edt_binary, utilities.runner_for(UtilityType::Ibcmd), utilities.runner_for(UtilityType::EdtCli), - ) - } - (SourceFormat::Edt, DumpMode::Full, BuilderBackend::Designer, _, Some(edt_binary)) => { - run_full_dump_edt_designer( + ), + (SourceFormat::Edt, DumpMode::Full, BuilderBackend::Designer, _, Some(edt_binary)) => { + run_full_dump_edt_designer( + context, + config, + &shadow_resolved, + location.path.as_path(), + edt_binary, + utilities.runner_for(UtilityType::V8), + utilities.runner_for(UtilityType::EdtCli), + ) + } + (SourceFormat::Edt, DumpMode::Full, BuilderBackend::Ibcmd, _, Some(edt_binary)) => { + run_full_dump_edt_ibcmd( + context, + config, + &shadow_resolved, + location.path.as_path(), + edt_binary, + utilities.runner_for(UtilityType::Ibcmd), + utilities.runner_for(UtilityType::EdtCli), + ) + } + ( + SourceFormat::Edt, + DumpMode::Partial, + BuilderBackend::Designer, + Some(objects), + Some(edt_binary), + ) => run_partial_dump_edt_designer( context, config, - &resolved, + &shadow_resolved, location.path.as_path(), edt_binary, utilities.runner_for(UtilityType::V8), utilities.runner_for(UtilityType::EdtCli), - ) - } - (SourceFormat::Edt, DumpMode::Full, BuilderBackend::Ibcmd, _, Some(edt_binary)) => { - run_full_dump_edt_ibcmd( + objects, + ), + ( + SourceFormat::Edt, + DumpMode::Partial, + BuilderBackend::Ibcmd, + Some(objects), + Some(edt_binary), + ) => run_partial_dump_edt_ibcmd( context, config, - &resolved, + &shadow_resolved, location.path.as_path(), edt_binary, utilities.runner_for(UtilityType::Ibcmd), utilities.runner_for(UtilityType::EdtCli), + objects, + ), + (_, DumpMode::Partial, _, None, _) => Err(AppError::Runtime( + "partial dump objects were not validated before execution".to_owned(), + )), + (SourceFormat::Edt, _, _, _, None) => Err(AppError::Runtime( + "EDT binary must be resolved before executing format=EDT dump".to_owned(), + )), + }; + let (platform_result, cleanup_message) = match command_result { + Ok(result) => result, + Err(error) => return Err(error), + }; + let dump = managed_manifest(configured_shadow_root, &[]) + .map_err(|error| AppError::Runtime(error.to_string()))?; + let merge = plan_manifest_merge(&baseline, &source, &dump); + failure_receipt = merge + .failed_receipt() + .map_err(|error| AppError::Runtime(error.to_string()))?; + if merge.has_conflicts() { + return Ok(PrivateDumpOutcome::Conflict( + merge + .conflict_receipt() + .map_err(|error| AppError::Runtime(error.to_string()))?, + )); + } + let writes = shadow_observation + .observe_writes(configured_shadow_root, &[], write_scope) + .map_err(|error| AppError::Runtime(error.to_string()))?; + let next_generation = observed_generation + .checked_add(1) + .ok_or_else(|| AppError::Runtime("runtime-state generation overflow".to_owned()))?; + let dump_transaction_id = DumpTransactionId::new(); + let prepared_publication = PublicationRequest::builder( + resolved.configured_source_context.path(), + configured_shadow_root, + &merge, + ) + .transaction_root(&publication_root) + .generation(next_generation) + .target_identity(publication_identity.clone()) + .dump_transaction_id(dump_transaction_id.clone()) + .prepare() + .map_err(|error| AppError::Runtime(error.to_string()))?; + run_before_source_publication_hook(); + if let Some(error) = interruption_before_publish(context, "dump source publication") { + return Err(rollback_publication_failure( + error, + &publication_root, + &resolved.configured_source_context, + &publication_identity, + observed_generation, + )); + } + let source_applied = prepared_publication + .apply() + .map_err(|error| AppError::Runtime(error.to_string()))?; + let post_publication = match crate::change_detection::analyzer::managed_inventory( + &resolved.configured_source_context, + ) { + Ok(inventory) => inventory, + Err(error) => { + return Err(rollback_publication_failure( + AppError::Runtime(error.to_string()), + &publication_root, + &resolved.configured_source_context, + &publication_identity, + observed_generation, + )) + } + }; + let prepared_state = + match prepare_dump_observation(&inventory.prepared, &post_publication, &merge) { + Ok(prepared) => prepared, + Err(error) => { + return Err(rollback_publication_failure( + error, + &publication_root, + &resolved.configured_source_context, + &publication_identity, + observed_generation, + )) + } + }; + let produced_cdfi = platform_shadow.path().join("ConfigDumpInfo.xml"); + let mut commit_request = + DumpStateCommitRequest::new(&prepared_state, configured_shadow_root, &produced_cdfi) + .with_transaction_id(dump_transaction_id.clone()); + if config.format == SourceFormat::Edt { + commit_request = commit_request.with_edt_platform_designer(platform_shadow.path()); + } + let visible_generation = match commit_dump_state_with_lock( + &resolved.configured_source_context, + &state_lock, + commit_request, + ) { + Ok(generation) => generation, + Err(commit_error) => { + return Err(rollback_publication_failure( + AppError::Runtime(commit_error.to_string()), + &publication_root, + &resolved.configured_source_context, + &publication_identity, + observed_generation, + )); + } + }; + if let Err(publication_error) = source_applied + .mark_state_visible(ObservedStateGeneration::with_dump_transaction( + visible_generation.value(), + dump_transaction_id.clone(), + )) + .and_then(|publication| publication.commit()) + { + recover_publication( + &publication_root, + resolved.configured_source_context.path(), + &publication_identity, + ObservedStateGeneration::with_dump_transaction( + visible_generation.value(), + dump_transaction_id, + ), ) + .map_err(|recovery_error| { + AppError::Runtime(format!( + "source publication finalization failed ({publication_error}); recovery also failed ({recovery_error})" + )) + })?; } - ( - SourceFormat::Edt, - DumpMode::Partial, - BuilderBackend::Designer, - Some(objects), - Some(edt_binary), - ) => run_partial_dump_edt_designer( - context, - config, - &resolved, - location.path.as_path(), - edt_binary, - utilities.runner_for(UtilityType::V8), - utilities.runner_for(UtilityType::EdtCli), - objects, - ), - ( - SourceFormat::Edt, - DumpMode::Partial, - BuilderBackend::Ibcmd, - Some(objects), - Some(edt_binary), - ) => run_partial_dump_edt_ibcmd( - context, - config, - &resolved, - location.path.as_path(), - edt_binary, - utilities.runner_for(UtilityType::Ibcmd), - utilities.runner_for(UtilityType::EdtCli), - objects, - ), - (_, DumpMode::Partial, _, None, _) => Err(AppError::Runtime( - "partial dump objects were not validated before execution".to_owned(), - )), - (SourceFormat::Edt, _, _, _, None) => Err(AppError::Runtime( - "EDT binary must be resolved before executing format=EDT dump".to_owned(), - )), - }; + Ok(PrivateDumpOutcome::Applied { + platform_result, + cleanup_message, + receipt: merge + .applied_receipt(&writes) + .map_err(|error| AppError::Runtime(error.to_string()))?, + }) + })(); drop(lock_guard); match result { - Ok((platform_result, cleanup_message)) => Ok(DumpResult { + Ok(PrivateDumpOutcome::Applied { + platform_result, + cleanup_message, + receipt, + }) => Ok(DumpResult { ok: true, source_set: Some(resolved.source_set_name), extension: resolved.extension, @@ -384,7 +815,26 @@ pub(super) fn run_dump_with_context( platform_log_path: platform_result.platform_log_path, duration_ms: started.elapsed().as_millis() as u64, message: cleanup_message.or_else(|| Some("dump completed successfully".to_owned())), + receipt, }), + Ok(PrivateDumpOutcome::Conflict(receipt)) => { + let message = "dump publication conflicts with local source changes".to_owned(); + Err(DumpExecutionFailure::with_payload( + AppError::Runtime(message.clone()), + DumpResult { + ok: false, + source_set: Some(resolved.source_set_name), + extension: resolved.extension, + selectors, + mode, + target_path: resolved.target_path, + platform_log_path: None, + duration_ms: started.elapsed().as_millis() as u64, + message: Some(message), + receipt, + }, + )) + } Err(error) => { let message = error.to_string(); Err(DumpExecutionFailure::with_payload( @@ -399,6 +849,7 @@ pub(super) fn run_dump_with_context( platform_log_path: None, duration_ms: started.elapsed().as_millis() as u64, message: Some(message), + receipt: failure_receipt, }, )) } diff --git a/src/use_cases/dump_config/helpers.rs b/src/use_cases/dump_config/helpers.rs index 881b706..80a1fe5 100644 --- a/src/use_cases/dump_config/helpers.rs +++ b/src/use_cases/dump_config/helpers.rs @@ -387,6 +387,7 @@ pub(super) fn empty_result( platform_log_path: None, duration_ms: started.elapsed().as_millis() as u64, message, + receipt: crate::domain::sync_receipt::SyncReceipt::empty_failed(), } } diff --git a/src/use_cases/dump_execution.rs b/src/use_cases/dump_execution.rs new file mode 100644 index 0000000..a01489f --- /dev/null +++ b/src/use_cases/dump_execution.rs @@ -0,0 +1,244 @@ +use std::collections::{BTreeMap, BTreeSet}; +use std::fs::{self, File, FileTimes}; +use std::io::{self, Read, Seek}; +use std::path::{Path, PathBuf}; +use std::time::{Duration, SystemTime}; + +use crate::use_cases::dump_shadow::{visit_managed_files, DumpShadowError, ManagedFileAccess}; +use sha2::{Digest, Sha256}; +use thiserror::Error; + +fn shadow_mtime_sentinel() -> SystemTime { + SystemTime::UNIX_EPOCH + Duration::from_secs(86_400) +} + +#[derive(Debug, Error)] +pub(crate) enum ShadowObservationError { + #[error("failed to inspect private dump shadow: {0}")] + Shadow(#[from] DumpShadowError), +} + +#[derive(Debug, Clone, PartialEq, Eq)] +struct ShadowFileState { + len: u64, + sha256: [u8; 32], + identity: StableFileIdentity, + mtime_ns: u128, +} + +#[derive(Debug, Clone, PartialEq, Eq)] +struct StableFileIdentity { + #[cfg(unix)] + device: u64, + #[cfg(unix)] + inode: u64, + #[cfg(windows)] + volume: Option, + #[cfg(windows)] + index: Option, +} + +/// Complete private-shadow observation captured before the platform command. +#[derive(Debug)] +pub(crate) struct ShadowObservation { + files: BTreeMap, +} + +/// Effective platform scope used to interpret observable private-shadow writes. +#[derive(Debug, Clone, Copy)] +pub(crate) enum EffectiveWriteScope<'a> { + Full { baseline: &'a BTreeSet }, + Incremental, +} + +/// Deterministic evidence of paths processed by the platform command. +#[derive(Debug, Clone, PartialEq, Eq)] +pub(crate) struct ShadowWriteSet { + paths: Vec, +} + +impl ShadowWriteSet { + pub(crate) fn paths(&self) -> &[String] { + &self.paths + } + + #[cfg(test)] + pub(crate) fn from_paths_for_test(paths: &[&str]) -> Self { + Self { + paths: paths.iter().map(|path| (*path).to_owned()).collect(), + } + } +} + +impl ShadowObservation { + pub(crate) fn normalize_and_capture( + root: &Path, + excluded_roots: &[PathBuf], + ) -> Result { + let mut files = BTreeMap::new(); + visit_managed_files( + root, + excluded_roots, + ManagedFileAccess::UpdateMetadata, + |relative, file| { + file.set_times(FileTimes::new().set_modified(shadow_mtime_sentinel()))?; + files.insert(relative.to_owned(), capture_open_file(file)?); + Ok(()) + }, + )?; + Ok(Self { files }) + } + + pub(crate) fn observe_writes( + self, + root: &Path, + excluded_roots: &[PathBuf], + scope: EffectiveWriteScope<'_>, + ) -> Result { + let mut after = BTreeMap::new(); + visit_managed_files( + root, + excluded_roots, + ManagedFileAccess::Read, + |relative, file| { + after.insert(relative.to_owned(), capture_open_file(file)?); + Ok(()) + }, + )?; + let mut paths = match scope { + EffectiveWriteScope::Full { baseline } => after + .keys() + .cloned() + .chain( + baseline + .iter() + .filter(|path| !after.contains_key(path.as_str())) + .cloned(), + ) + .collect::>(), + EffectiveWriteScope::Incremental => self + .files + .keys() + .chain(after.keys()) + .filter(|path| self.files.get(*path) != after.get(*path)) + .cloned() + .collect(), + }; + Ok(ShadowWriteSet { + paths: std::mem::take(&mut paths).into_iter().collect(), + }) + } +} + +fn capture_open_file(file: &mut File) -> io::Result { + file.rewind()?; + let metadata = file.metadata()?; + if !metadata.is_file() { + return Err(io::Error::new( + io::ErrorKind::InvalidData, + "managed entry is not a regular file", + )); + } + let mut hasher = Sha256::new(); + let mut buffer = [0_u8; 64 * 1024]; + loop { + let read = file.read(&mut buffer)?; + if read == 0 { + break; + } + hasher.update(&buffer[..read]); + } + let modified = metadata + .modified()? + .duration_since(SystemTime::UNIX_EPOCH) + .map_err(|_| io::Error::new(io::ErrorKind::InvalidData, "mtime predates Unix epoch"))? + .as_nanos(); + Ok(ShadowFileState { + len: metadata.len(), + sha256: hasher.finalize().into(), + identity: stable_identity(&metadata), + mtime_ns: modified, + }) +} + +#[cfg(unix)] +fn stable_identity(metadata: &fs::Metadata) -> StableFileIdentity { + use std::os::unix::fs::MetadataExt; + StableFileIdentity { + device: metadata.dev(), + inode: metadata.ino(), + } +} + +#[cfg(windows)] +fn stable_identity(metadata: &fs::Metadata) -> StableFileIdentity { + use std::os::windows::fs::MetadataExt; + StableFileIdentity { + volume: metadata.volume_serial_number(), + index: metadata.file_index(), + } +} + +#[cfg(not(any(unix, windows)))] +fn stable_identity(_metadata: &fs::Metadata) -> StableFileIdentity { + StableFileIdentity {} +} + +#[cfg(test)] +mod tests { + use std::collections::BTreeSet; + use std::fs; + + use tempfile::tempdir; + + use super::{EffectiveWriteScope, ShadowObservation}; + + #[test] + fn unchanged_rewrite_is_observable_but_untouched_file_is_not() { + let dir = tempdir().expect("tempdir"); + fs::write(dir.path().join("rewritten.txt"), b"same").expect("rewritten"); + fs::write(dir.path().join("untouched.txt"), b"same").expect("untouched"); + let before = ShadowObservation::normalize_and_capture(dir.path(), &[]).expect("before"); + + fs::write(dir.path().join("rewritten.txt"), b"same").expect("rewrite"); + let writes = before + .observe_writes(dir.path(), &[], EffectiveWriteScope::Incremental) + .expect("writes"); + + assert_eq!(writes.paths(), &["rewritten.txt"]); + } + + #[test] + fn full_scope_contains_complete_final_tree_and_baseline_deletions() { + let dir = tempdir().expect("tempdir"); + fs::write(dir.path().join("kept.txt"), b"same").expect("kept"); + let before = ShadowObservation::normalize_and_capture(dir.path(), &[]).expect("before"); + let baseline = BTreeSet::from(["deleted.txt".to_owned(), "kept.txt".to_owned()]); + + let writes = before + .observe_writes( + dir.path(), + &[], + EffectiveWriteScope::Full { + baseline: &baseline, + }, + ) + .expect("writes"); + + assert_eq!(writes.paths(), &["deleted.txt", "kept.txt"]); + } + + #[test] + fn platform_owned_cdfi_is_excluded_from_write_evidence() { + let dir = tempdir().expect("tempdir"); + fs::write(dir.path().join("ConfigDumpInfo.xml"), b"before").expect("cdfi"); + let before = ShadowObservation::normalize_and_capture(dir.path(), &[]).expect("before"); + fs::write(dir.path().join("ConfigDumpInfo.xml"), b"after").expect("cdfi rewrite"); + + let writes = before + .observe_writes(dir.path(), &[], EffectiveWriteScope::Incremental) + .expect("writes"); + + assert!(writes.paths().is_empty()); + } +} diff --git a/src/use_cases/dump_shadow.rs b/src/use_cases/dump_shadow.rs new file mode 100644 index 0000000..c99fb3a --- /dev/null +++ b/src/use_cases/dump_shadow.rs @@ -0,0 +1,1566 @@ +use std::collections::{BTreeMap, BTreeSet}; +use std::fs::{self, OpenOptions}; +use std::io::{self, Read, Write}; +use std::path::{Component, Path, PathBuf}; + +use serde::{Deserialize, Serialize}; +use sha2::{Digest, Sha256}; +use tempfile::TempDir; +use thiserror::Error; +use walkdir::WalkDir; + +use crate::change_detection::scanner::{ + is_always_ignored_relative_path, portable_relative_path, ScanError, SourceInventoryPolicy, +}; +use crate::domain::runtime_state::{BaselineRole, IbBaseline, StateGeneration}; +use crate::domain::source_set::SourceSetContext; +use crate::use_cases::request::DumpModeRequest; +use crate::use_cases::runtime_state::{inspect_private_cdfi, PrivateCdfiState, RuntimeStateError}; + +pub(crate) type BaselineFileHash = [u8; 32]; + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum ManagedFileAccess { + Read, + UpdateMetadata, +} + +const BASELINE_MANIFEST_VERSION: u32 = 1; +const BASELINE_MANIFEST_NAME: &str = "manifest.json"; +const BASELINE_FILES_NAME: &str = "files"; + +#[cfg(test)] +thread_local! { + static AFTER_INVENTORY_HOOK: std::cell::RefCell>> = + const { std::cell::RefCell::new(None) }; + static BEFORE_BASELINE_SEED_COPY: std::cell::RefCell>> = + const { std::cell::RefCell::new(None) }; +} + +#[cfg(test)] +fn set_after_inventory_hook(hook: impl FnOnce() + 'static) { + AFTER_INVENTORY_HOOK.with(|slot| slot.replace(Some(Box::new(hook)))); +} + +#[cfg(test)] +fn run_after_inventory_hook() { + AFTER_INVENTORY_HOOK.with(|slot| { + if let Some(hook) = slot.take() { + hook(); + } + }); +} + +#[cfg(not(test))] +fn run_after_inventory_hook() {} + +#[cfg(test)] +fn run_before_baseline_seed_copy() { + BEFORE_BASELINE_SEED_COPY.with(|slot| { + if let Some(hook) = slot.take() { + hook(); + } + }); +} + +#[cfg(test)] +fn set_before_baseline_seed_copy(hook: impl FnOnce() + 'static) { + BEFORE_BASELINE_SEED_COPY.with(|slot| slot.replace(Some(Box::new(hook)))); +} + +#[cfg(not(test))] +fn run_before_baseline_seed_copy() {} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum EffectiveDumpMode { + Full, + Incremental, + Partial, +} + +#[derive(Debug)] +pub(crate) enum BaselineInspection { + Missing, + Valid(ValidatedBaseline), + Corrupt(String), +} + +#[derive(Debug)] +pub(crate) struct ValidatedBaseline { + files_root: PathBuf, + files: Vec, +} + +impl ValidatedBaseline { + pub(crate) fn files_root(&self) -> &Path { + &self.files_root + } + + pub(crate) fn files(&self) -> &[ValidatedBaselineFile] { + &self.files + } +} + +#[derive(Debug, Clone, PartialEq, Eq)] +pub(crate) struct ValidatedBaselineFile { + path: String, + len: u64, + sha256: BaselineFileHash, +} + +impl ValidatedBaselineFile { + pub(crate) fn path(&self) -> &str { + &self.path + } + + #[cfg(test)] + pub(crate) const fn byte_len(&self) -> u64 { + self.len + } + + pub(crate) const fn sha256(&self) -> BaselineFileHash { + self.sha256 + } +} + +#[derive(Debug, Error)] +pub(crate) enum DumpShadowError { + #[cfg(not(any(unix, windows)))] + #[error("safe descriptor-relative dump shadow access is unavailable on this platform")] + UnsupportedSafeFilesystem, + #[error("failed to inspect dump baseline '{path}': {source}")] + Inspect { path: PathBuf, source: io::Error }, + #[error("failed to access dump shadow '{path}': {source}")] + Io { path: PathBuf, source: io::Error }, + #[error("invalid managed source inventory: {0}")] + Inventory(#[from] ScanError), + #[error("failed to walk dump baseline: {0}")] + Walk(#[from] walkdir::Error), + #[error("failed to serialize dump baseline manifest: {0}")] + Serialize(#[from] serde_json::Error), + #[error("failed to inspect private CDFI: {0}")] + Cdfi(#[from] RuntimeStateError), + #[error("dump baseline already exists: '{0}'")] + AlreadyExists(PathBuf), +} + +#[derive(Debug, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +struct BaselineManifest { + version: u32, + files: Vec, +} + +#[derive(Debug, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +struct BaselineFile { + path: String, + len: u64, + sha256: String, +} + +#[cfg(test)] +pub(crate) fn publish_complete_baseline( + source_root: &Path, + excluded_roots: &[PathBuf], + handle: &IbBaseline, +) -> Result<(), DumpShadowError> { + stage_complete_baseline(source_root, excluded_roots, handle.path()) +} + +/// Materializes a complete baseline at an owned, currently absent destination. +/// +/// Runtime-state publication uses this to stage the directory inside its journaled +/// transaction before the generation becomes visible in redb. +pub(crate) fn stage_complete_baseline( + source_root: &Path, + excluded_roots: &[PathBuf], + destination: &Path, +) -> Result<(), DumpShadowError> { + #[cfg(not(any(unix, windows)))] + return Err(DumpShadowError::UnsupportedSafeFilesystem); + #[cfg(any(unix, windows))] + { + let (source_root, excluded_roots) = canonical_inventory_scope(source_root, excluded_roots)?; + match fs::symlink_metadata(destination) { + Ok(_) => return Err(DumpShadowError::AlreadyExists(destination.to_path_buf())), + Err(source) if source.kind() == io::ErrorKind::NotFound => {} + Err(source) => { + return Err(DumpShadowError::Inspect { + path: destination.to_path_buf(), + source, + }) + } + } + + let parent = destination.parent().ok_or_else(|| DumpShadowError::Io { + path: destination.to_path_buf(), + source: io::Error::new(io::ErrorKind::InvalidInput, "baseline has no parent"), + })?; + fs::create_dir_all(parent).map_err(|source| io_error(parent, source))?; + let transaction = tempfile::Builder::new() + .prefix("baseline-") + .tempdir_in(parent) + .map_err(|source| io_error(parent, source))?; + let files_root = transaction.path().join(BASELINE_FILES_NAME); + fs::create_dir(&files_root).map_err(|source| io_error(&files_root, source))?; + + let inventory = managed_paths(&source_root, &excluded_roots)?; + run_after_inventory_hook(); + let mut files = Vec::with_capacity(inventory.len()); + for (relative_path, source_path) in inventory { + let relative = Path::new(&relative_path); + let target = files_root.join(relative); + if let Some(target_parent) = target.parent() { + fs::create_dir_all(target_parent) + .map_err(|source| io_error(target_parent, source))?; + } + copy_regular_no_follow(&source_root, relative, &source_path, &target)?; + let (copied_len, copied_hash) = + hash_regular_file(&target).map_err(|error| regular_error(&target, error))?; + files.push(BaselineFile { + path: relative_path, + len: copied_len, + sha256: hex_hash(copied_hash), + }); + } + files.sort_by(|left, right| left.path.cmp(&right.path)); + let manifest = BaselineManifest { + version: BASELINE_MANIFEST_VERSION, + files, + }; + let manifest_path = transaction.path().join(BASELINE_MANIFEST_NAME); + let bytes = serde_json::to_vec(&manifest)?; + write_new_synced(&manifest_path, &bytes)?; + sync_directory(&files_root)?; + sync_directory(transaction.path())?; + + let transaction_path = transaction.keep(); + if let Err(source) = fs::rename(&transaction_path, destination) { + let _ = fs::remove_dir_all(&transaction_path); + return Err(io_error(destination, source)); + } + sync_directory(parent)?; + Ok(()) + } +} + +fn managed_paths( + source_root: &Path, + excluded_roots: &[PathBuf], +) -> Result, DumpShadowError> { + let policy = SourceInventoryPolicy::new(source_root, excluded_roots)?; + let mut files = Vec::new(); + for entry in WalkDir::new(source_root) + .follow_links(false) + .into_iter() + .filter_entry(|entry| !entry.file_type().is_dir() || policy.should_descend(entry.path())) + { + let entry = entry?; + if !entry.file_type().is_file() || !policy.includes_file(entry.path()) { + continue; + } + files.push(( + portable_relative_path(source_root, entry.path())?, + entry.path().to_path_buf(), + )); + } + files.sort_by(|left, right| left.0.cmp(&right.0)); + Ok(files) +} + +/// Scan one complete managed tree into a deterministic byte-exact manifest. +/// +/// The inventory policy is shared with baseline publication, so platform-owned CDFI, +/// excluded roots and symlinks cannot enter a three-way merge accidentally. +pub(crate) fn managed_manifest( + source_root: &Path, + excluded_roots: &[PathBuf], +) -> Result, DumpShadowError> { + #[cfg(not(any(unix, windows)))] + return Err(DumpShadowError::UnsupportedSafeFilesystem); + #[cfg(any(unix, windows))] + { + let (source_root, excluded_roots) = canonical_inventory_scope(source_root, excluded_roots)?; + let inventory = managed_paths(&source_root, &excluded_roots)?; + run_after_inventory_hook(); + inventory + .into_iter() + .map(|(relative_path, source_path)| { + #[cfg(unix)] + let observation = hash_regular_beneath_unix( + &source_root, + Path::new(&relative_path), + &source_path, + ); + #[cfg(windows)] + let observation = hash_regular_beneath_windows( + &source_root, + Path::new(&relative_path), + &source_path, + ); + let (_, hash) = observation.map_err(|error| regular_error(&source_path, error))?; + Ok((relative_path, hash)) + }) + .collect() + } +} + +pub(crate) fn visit_managed_files( + root: &Path, + excluded_roots: &[PathBuf], + access: ManagedFileAccess, + mut visitor: impl FnMut(&str, &mut fs::File) -> io::Result<()>, +) -> Result<(), DumpShadowError> { + let (root, excluded_roots) = canonical_inventory_scope(root, excluded_roots)?; + let inventory = managed_paths(&root, &excluded_roots)?; + run_after_inventory_hook(); + for (relative, display) in inventory { + let mut file = open_managed_file(&root, Path::new(&relative), &display, access)?; + visitor(&relative, &mut file).map_err(|source| io_error(&display, source))?; + } + Ok(()) +} + +fn canonical_inventory_scope( + root: &Path, + excluded_roots: &[PathBuf], +) -> Result<(PathBuf, Vec), DumpShadowError> { + let canonical = fs::canonicalize(root).map_err(|source| io_error(root, source))?; + let metadata = + fs::symlink_metadata(&canonical).map_err(|source| io_error(&canonical, source))?; + if !metadata.file_type().is_dir() || metadata.file_type().is_symlink() { + return Err(io_error( + root, + io::Error::new( + io::ErrorKind::InvalidInput, + "managed root is not a directory", + ), + )); + } + let excluded = excluded_roots + .iter() + .map(|excluded| { + excluded + .strip_prefix(root) + .map_or_else(|_| excluded.clone(), |relative| canonical.join(relative)) + }) + .collect(); + Ok((canonical, excluded)) +} + +fn open_managed_file( + root: &Path, + relative: &Path, + display: &Path, + access: ManagedFileAccess, +) -> Result { + #[cfg(unix)] + { + let _ = access; + return open_regular_beneath_unix(root, relative, display); + } + #[cfg(windows)] + { + let relative = relative.to_str().ok_or_else(|| { + io_error( + display, + io::Error::new(io::ErrorKind::InvalidInput, "managed path is not UTF-8"), + ) + })?; + let parent = crate::support::windows_fs::open_parent(root, relative, false) + .map_err(|source| io_error(display, source))? + .ok_or_else(|| io_error(display, io::Error::from(io::ErrorKind::NotFound)))?; + return match access { + ManagedFileAccess::Read => { + crate::support::windows_fs::open_regular_existing(&parent, false) + } + ManagedFileAccess::UpdateMetadata => { + crate::support::windows_fs::open_regular_existing_for_metadata(&parent) + } + } + .map_err(|source| io_error(display, source)); + } + #[cfg(not(any(unix, windows)))] + { + let _ = (root, relative, display, access); + Err(DumpShadowError::UnsupportedSafeFilesystem) + } +} + +pub(crate) fn inspect_baseline(handle: &IbBaseline) -> Result { + inspect_baseline_path(handle.path()) +} + +pub(crate) fn inspect_baseline_path( + baseline_path: &Path, +) -> Result { + let root_metadata = match fs::symlink_metadata(baseline_path) { + Ok(metadata) => metadata, + Err(source) if source.kind() == io::ErrorKind::NotFound => { + return Ok(BaselineInspection::Missing) + } + Err(source) => { + return Err(DumpShadowError::Inspect { + path: baseline_path.to_path_buf(), + source, + }) + } + }; + if root_metadata.file_type().is_symlink() || !root_metadata.file_type().is_dir() { + return Ok(BaselineInspection::Corrupt( + "baseline root is not a regular directory".to_owned(), + )); + } + + let manifest_path = baseline_path.join(BASELINE_MANIFEST_NAME); + let manifest_bytes = match read_regular_file(&manifest_path) { + Ok(bytes) => bytes, + Err(ReadRegularError::Invalid(reason)) => return Ok(BaselineInspection::Corrupt(reason)), + Err(ReadRegularError::Io(source)) if source.kind() == io::ErrorKind::NotFound => { + return Ok(BaselineInspection::Corrupt( + "manifest is missing".to_owned(), + )) + } + Err(ReadRegularError::Io(source)) => return Err(io_error(&manifest_path, source)), + }; + let manifest: BaselineManifest = match serde_json::from_slice(&manifest_bytes) { + Ok(manifest) => manifest, + Err(error) => { + return Ok(BaselineInspection::Corrupt(format!( + "manifest is malformed: {error}" + ))) + } + }; + if manifest.version != BASELINE_MANIFEST_VERSION { + return Ok(BaselineInspection::Corrupt(format!( + "unsupported manifest version {}", + manifest.version + ))); + } + if !manifest + .files + .windows(2) + .all(|pair| pair[0].path < pair[1].path) + { + return Ok(BaselineInspection::Corrupt( + "manifest paths are not strictly sorted".to_owned(), + )); + } + + let files_root = baseline_path.join(BASELINE_FILES_NAME); + let files_metadata = match fs::symlink_metadata(&files_root) { + Ok(metadata) => metadata, + Err(source) if source.kind() == io::ErrorKind::NotFound => { + return Ok(BaselineInspection::Corrupt( + "files root is missing".to_owned(), + )) + } + Err(source) => return Err(io_error(&files_root, source)), + }; + if files_metadata.file_type().is_symlink() || !files_metadata.file_type().is_dir() { + return Ok(BaselineInspection::Corrupt( + "files root is not a regular directory".to_owned(), + )); + } + + let mut actual_paths = BTreeSet::new(); + for entry in WalkDir::new(&files_root).follow_links(false) { + let entry = entry?; + if entry.path() == files_root { + continue; + } + if entry.file_type().is_dir() { + continue; + } + if !entry.file_type().is_file() || entry.file_type().is_symlink() { + return Ok(BaselineInspection::Corrupt(format!( + "baseline contains non-regular entry '{}'", + entry.path().display() + ))); + } + actual_paths.insert(portable_relative_path(&files_root, entry.path())?); + } + + let expected_paths = manifest + .files + .iter() + .map(|file| file.path.clone()) + .collect::>(); + if actual_paths != expected_paths { + return Ok(BaselineInspection::Corrupt( + "manifest file set differs from baseline files".to_owned(), + )); + } + let mut validated_files = Vec::with_capacity(manifest.files.len()); + for file in &manifest.files { + if !is_safe_manifest_path(&file.path) + || is_cdfi_path(&file.path) + || is_always_ignored_relative_path(Path::new(&file.path)) + { + return Ok(BaselineInspection::Corrupt(format!( + "manifest contains invalid managed path '{}'", + file.path + ))); + } + let path = files_root.join(&file.path); + let (observed_len, hash) = match hash_regular_file(&path) { + Ok(observation) => observation, + Err(ReadRegularError::Invalid(reason)) => { + return Ok(BaselineInspection::Corrupt(reason)) + } + Err(ReadRegularError::Io(source)) => return Err(io_error(&path, source)), + }; + let Some(expected_hash) = decode_sha256(&file.sha256) else { + return Ok(BaselineInspection::Corrupt(format!( + "baseline file '{}' has invalid SHA-256", + file.path + ))); + }; + if observed_len != file.len || hash != expected_hash { + return Ok(BaselineInspection::Corrupt(format!( + "baseline file '{}' does not match manifest", + file.path + ))); + } + validated_files.push(ValidatedBaselineFile { + path: file.path.clone(), + len: file.len, + sha256: expected_hash, + }); + } + + Ok(BaselineInspection::Valid(ValidatedBaseline { + files_root, + files: validated_files, + })) +} + +pub(crate) struct DumpShadow { + _transaction: TempDir, + path: PathBuf, + mode: EffectiveDumpMode, +} + +impl DumpShadow { + pub(crate) fn prepare( + context: &SourceSetContext, + role: BaselineRole, + generation: StateGeneration, + requested: DumpModeRequest, + ) -> Result { + let transactions_dir = context.transactions_dir(); + fs::create_dir_all(&transactions_dir) + .map_err(|source| io_error(&transactions_dir, source))?; + let transaction = tempfile::Builder::new() + .prefix("dump-shadow-") + .tempdir_in(&transactions_dir) + .map_err(|source| io_error(&transactions_dir, source))?; + let path = transaction.path().join("shadow"); + fs::create_dir(&path).map_err(|source| io_error(&path, source))?; + + let requested_mode = match requested { + DumpModeRequest::Full => EffectiveDumpMode::Full, + DumpModeRequest::Incremental => EffectiveDumpMode::Incremental, + DumpModeRequest::Partial => EffectiveDumpMode::Partial, + }; + if requested_mode == EffectiveDumpMode::Full { + return Ok(Self { + _transaction: transaction, + path, + mode: EffectiveDumpMode::Full, + }); + } + + let baseline = inspect_baseline(&context.baseline(role, generation))?; + let cdfi = inspect_private_cdfi(&context.private_cdfi_path())?; + match baseline { + BaselineInspection::Missing => Ok(Self::full(transaction, path)), + BaselineInspection::Corrupt(_reason) => Ok(Self::full(transaction, path)), + BaselineInspection::Valid(baseline) => match cdfi { + PrivateCdfiState::Missing => Ok(Self::full(transaction, path)), + PrivateCdfiState::Corrupt(_reason) => Ok(Self::full(transaction, path)), + PrivateCdfiState::Valid(cdfi) => { + run_before_baseline_seed_copy(); + let baseline_still_valid = baseline.files_root().parent().is_some_and(|root| { + matches!( + inspect_baseline_path(root), + Ok(BaselineInspection::Valid(_)) + ) + }); + if !baseline_still_valid { + return Ok(Self::full(transaction, path)); + } + if copy_validated_baseline(&baseline, &path).is_err() { + fs::remove_dir_all(&path).map_err(|source| io_error(&path, source))?; + fs::create_dir(&path).map_err(|source| io_error(&path, source))?; + return Ok(Self::full(transaction, path)); + } + write_new_synced(&path.join("ConfigDumpInfo.xml"), cdfi.bytes())?; + Ok(Self { + _transaction: transaction, + path, + mode: requested_mode, + }) + } + }, + } + } + + pub(crate) fn path(&self) -> &Path { + &self.path + } + + pub(crate) const fn mode(&self) -> EffectiveDumpMode { + self.mode + } + + fn full(transaction: TempDir, path: PathBuf) -> Self { + Self { + _transaction: transaction, + path, + mode: EffectiveDumpMode::Full, + } + } +} + +fn copy_validated_baseline( + baseline: &ValidatedBaseline, + target_root: &Path, +) -> Result<(), DumpShadowError> { + for file in baseline.files() { + let relative = Path::new(file.path()); + let source = baseline.files_root().join(relative); + let target = target_root.join(relative); + if let Some(parent) = target.parent() { + fs::create_dir_all(parent).map_err(|source| io_error(parent, source))?; + } + copy_regular_no_follow(baseline.files_root(), relative, &source, &target)?; + let (len, sha256) = + hash_regular_file(&target).map_err(|error| regular_error(&target, error))?; + if len != file.len || sha256 != file.sha256 { + return Err(io_error( + &source, + io::Error::new( + io::ErrorKind::InvalidData, + "baseline changed after validation", + ), + )); + } + } + Ok(()) +} + +fn is_safe_manifest_path(path: &str) -> bool { + !path.is_empty() + && !path.contains('\\') + && Path::new(path) + .components() + .all(|component| matches!(component, Component::Normal(_))) +} + +fn is_cdfi_path(path: &str) -> bool { + Path::new(path) + .file_name() + .and_then(|name| name.to_str()) + .is_some_and(|name| name.eq_ignore_ascii_case("ConfigDumpInfo.xml")) +} + +fn decode_sha256(value: &str) -> Option { + if value.len() != 64 { + return None; + } + let mut result = [0_u8; 32]; + for (index, chunk) in value.as_bytes().chunks_exact(2).enumerate() { + let encoded = std::str::from_utf8(chunk).ok()?; + result[index] = u8::from_str_radix(encoded, 16).ok()?; + } + Some(result) +} + +enum ReadRegularError { + Invalid(String), + Io(io::Error), +} + +fn read_regular_file(path: &Path) -> Result, ReadRegularError> { + let metadata = fs::symlink_metadata(path).map_err(ReadRegularError::Io)?; + if metadata.file_type().is_symlink() || !metadata.file_type().is_file() { + return Err(ReadRegularError::Invalid(format!( + "'{}' is not a regular file", + path.display() + ))); + } + fs::read(path).map_err(ReadRegularError::Io) +} + +fn hash_regular_file(path: &Path) -> Result<(u64, BaselineFileHash), ReadRegularError> { + let metadata = fs::symlink_metadata(path).map_err(ReadRegularError::Io)?; + if metadata.file_type().is_symlink() || !metadata.file_type().is_file() { + return Err(ReadRegularError::Invalid(format!( + "'{}' is not a regular file", + path.display() + ))); + } + let mut options = OpenOptions::new(); + options.read(true); + #[cfg(unix)] + { + use std::os::unix::fs::OpenOptionsExt; + options.custom_flags(libc::O_NOFOLLOW | libc::O_CLOEXEC); + } + #[cfg(windows)] + { + use std::os::windows::fs::OpenOptionsExt; + const FILE_FLAG_OPEN_REPARSE_POINT: u32 = 0x0020_0000; + options.custom_flags(FILE_FLAG_OPEN_REPARSE_POINT); + } + let mut file = options.open(path).map_err(ReadRegularError::Io)?; + let opened = file.metadata().map_err(ReadRegularError::Io)?; + #[cfg(windows)] + let is_reparse_point = { + use std::os::windows::fs::MetadataExt; + const FILE_ATTRIBUTE_REPARSE_POINT: u32 = 0x0000_0400; + opened.file_attributes() & FILE_ATTRIBUTE_REPARSE_POINT != 0 + }; + #[cfg(not(windows))] + let is_reparse_point = false; + if !opened.file_type().is_file() || is_reparse_point { + return Err(ReadRegularError::Invalid(format!( + "'{}' is not a regular file", + path.display() + ))); + } + let mut digest = Sha256::new(); + let mut length = 0_u64; + let mut buffer = [0_u8; 64 * 1024]; + loop { + let read = file.read(&mut buffer).map_err(ReadRegularError::Io)?; + if read == 0 { + break; + } + length = length.saturating_add(read as u64); + digest.update(&buffer[..read]); + } + Ok((length, digest.finalize().into())) +} + +#[cfg(windows)] +fn hash_open_file(file: &mut fs::File) -> Result<(u64, BaselineFileHash), ReadRegularError> { + let mut digest = Sha256::new(); + let mut length = 0_u64; + let mut buffer = [0_u8; 64 * 1024]; + loop { + let read = file.read(&mut buffer).map_err(ReadRegularError::Io)?; + if read == 0 { + break; + } + length = length.saturating_add(read as u64); + digest.update(&buffer[..read]); + } + Ok((length, digest.finalize().into())) +} + +#[cfg(unix)] +fn hash_regular_beneath_unix( + source_root: &Path, + relative: &Path, + source: &Path, +) -> Result<(u64, BaselineFileHash), ReadRegularError> { + let mut file = + open_regular_beneath_unix(source_root, relative, source).map_err(|error| match error { + DumpShadowError::Io { source, .. } | DumpShadowError::Inspect { source, .. } => { + ReadRegularError::Io(source) + } + other => ReadRegularError::Invalid(other.to_string()), + })?; + let mut digest = Sha256::new(); + let mut length = 0_u64; + let mut buffer = [0_u8; 64 * 1024]; + loop { + let read = file.read(&mut buffer).map_err(ReadRegularError::Io)?; + if read == 0 { + break; + } + length = length.saturating_add(read as u64); + digest.update(&buffer[..read]); + } + Ok((length, digest.finalize().into())) +} + +fn regular_error(path: &Path, error: ReadRegularError) -> DumpShadowError { + match error { + ReadRegularError::Invalid(reason) => { + io_error(path, io::Error::new(io::ErrorKind::InvalidData, reason)) + } + ReadRegularError::Io(source) => io_error(path, source), + } +} + +fn hex_hash(hash: BaselineFileHash) -> String { + const HEX: &[u8; 16] = b"0123456789abcdef"; + let mut encoded = String::with_capacity(64); + for byte in hash { + encoded.push(HEX[(byte >> 4) as usize] as char); + encoded.push(HEX[(byte & 0x0f) as usize] as char); + } + encoded +} + +fn write_new_synced(path: &Path, bytes: &[u8]) -> Result<(), DumpShadowError> { + let mut file = OpenOptions::new() + .write(true) + .create_new(true) + .open(path) + .map_err(|source| io_error(path, source))?; + file.write_all(bytes) + .map_err(|source| io_error(path, source))?; + file.sync_all().map_err(|source| io_error(path, source)) +} + +fn copy_regular_no_follow( + source_root: &Path, + relative: &Path, + source: &Path, + target: &Path, +) -> Result<(), DumpShadowError> { + #[cfg(unix)] + return copy_regular_beneath_unix(source_root, relative, source, target); + #[cfg(windows)] + return copy_regular_beneath_windows(source_root, relative, source, target); + #[cfg(not(any(unix, windows)))] + Err(DumpShadowError::UnsupportedSafeFilesystem) +} + +#[cfg(windows)] +fn open_regular_beneath_windows( + source_root: &Path, + relative: &Path, + source: &Path, +) -> Result { + let relative = relative.to_str().ok_or_else(|| { + io_error( + source, + io::Error::new(io::ErrorKind::InvalidInput, "managed path is not UTF-8"), + ) + })?; + let parent = crate::support::windows_fs::open_parent(source_root, relative, false) + .map_err(|error| io_error(source, error))? + .ok_or_else(|| io_error(source, io::Error::from(io::ErrorKind::NotFound)))?; + crate::support::windows_fs::open_regular_existing(&parent, false) + .map_err(|error| io_error(source, error)) +} + +#[cfg(windows)] +fn hash_regular_beneath_windows( + source_root: &Path, + relative: &Path, + source: &Path, +) -> Result<(u64, BaselineFileHash), ReadRegularError> { + let mut file = open_regular_beneath_windows(source_root, relative, source).map_err( + |error| match error { + DumpShadowError::Io { source, .. } | DumpShadowError::Inspect { source, .. } => { + ReadRegularError::Io(source) + } + other => ReadRegularError::Invalid(other.to_string()), + }, + )?; + hash_open_file(&mut file) +} + +#[cfg(windows)] +fn copy_regular_beneath_windows( + source_root: &Path, + relative: &Path, + source: &Path, + target: &Path, +) -> Result<(), DumpShadowError> { + let mut input = open_regular_beneath_windows(source_root, relative, source)?; + let before = input.metadata().map_err(|error| io_error(source, error))?; + let mut output = OpenOptions::new() + .write(true) + .create_new(true) + .open(target) + .map_err(|error| io_error(target, error))?; + io::copy(&mut input, &mut output).map_err(|error| io_error(target, error))?; + output.sync_all().map_err(|error| io_error(target, error))?; + let after = input.metadata().map_err(|error| io_error(source, error))?; + use std::os::windows::fs::MetadataExt; + if before.volume_serial_number() != after.volume_serial_number() + || before.file_index() != after.file_index() + || before.file_size() != after.file_size() + { + return Err(io_error( + source, + io::Error::new(io::ErrorKind::InvalidData, "source changed during copy"), + )); + } + Ok(()) +} + +#[cfg(unix)] +fn copy_regular_beneath_unix( + source_root: &Path, + relative: &Path, + source: &Path, + target: &Path, +) -> Result<(), DumpShadowError> { + let mut input = open_regular_beneath_unix(source_root, relative, source)?; + let before = input.metadata().map_err(|error| io_error(source, error))?; + if !before.file_type().is_file() { + return Err(io_error( + source, + io::Error::new(io::ErrorKind::InvalidData, "source is not a regular file"), + )); + } + let mut output = OpenOptions::new() + .write(true) + .create_new(true) + .open(target) + .map_err(|error| io_error(target, error))?; + io::copy(&mut input, &mut output).map_err(|error| io_error(target, error))?; + output.sync_all().map_err(|error| io_error(target, error))?; + let after = input.metadata().map_err(|error| io_error(source, error))?; + use std::os::unix::fs::MetadataExt; + if before.dev() != after.dev() || before.ino() != after.ino() || before.len() != after.len() { + return Err(io_error( + source, + io::Error::new(io::ErrorKind::InvalidData, "source changed during copy"), + )); + } + Ok(()) +} + +#[cfg(unix)] +fn open_regular_beneath_unix( + source_root: &Path, + relative: &Path, + source: &Path, +) -> Result { + use std::ffi::CString; + use std::os::fd::{AsRawFd, FromRawFd}; + use std::os::unix::ffi::OsStrExt; + use std::os::unix::fs::OpenOptionsExt; + + let mut root_options = OpenOptions::new(); + root_options + .read(true) + .custom_flags(libc::O_DIRECTORY | libc::O_NOFOLLOW | libc::O_CLOEXEC); + let mut directory = root_options + .open(source_root) + .map_err(|source_error| io_error(source_root, source_error))?; + let components = relative.components().collect::>(); + let Some((file_component, parent_components)) = components.split_last() else { + return Err(io_error( + source, + io::Error::new(io::ErrorKind::InvalidInput, "empty relative path"), + )); + }; + for component in parent_components { + let Component::Normal(name) = component else { + return Err(io_error( + source, + io::Error::new(io::ErrorKind::InvalidInput, "non-normal path component"), + )); + }; + let name = CString::new(name.as_bytes()).map_err(|_| { + io_error( + source, + io::Error::new(io::ErrorKind::InvalidInput, "path contains NUL"), + ) + })?; + // SAFETY: the descriptor is live and each single component is opened no-follow. + let descriptor = unsafe { + libc::openat( + directory.as_raw_fd(), + name.as_ptr(), + libc::O_RDONLY | libc::O_DIRECTORY | libc::O_NOFOLLOW | libc::O_CLOEXEC, + ) + }; + if descriptor < 0 { + return Err(io_error(source, io::Error::last_os_error())); + } + // SAFETY: openat returned a new owned descriptor. + directory = unsafe { fs::File::from_raw_fd(descriptor) }; + } + let Component::Normal(file_name) = file_component else { + return Err(io_error( + source, + io::Error::new(io::ErrorKind::InvalidInput, "non-normal file component"), + )); + }; + let file_name = CString::new(file_name.as_bytes()).map_err(|_| { + io_error( + source, + io::Error::new(io::ErrorKind::InvalidInput, "path contains NUL"), + ) + })?; + // SAFETY: same descriptor/name invariants as the directory traversal above. + let descriptor = unsafe { + libc::openat( + directory.as_raw_fd(), + file_name.as_ptr(), + libc::O_RDONLY | libc::O_NOFOLLOW | libc::O_CLOEXEC, + ) + }; + if descriptor < 0 { + return Err(io_error(source, io::Error::last_os_error())); + } + // SAFETY: openat returned a new owned descriptor. + let input = unsafe { fs::File::from_raw_fd(descriptor) }; + if !input + .metadata() + .map_err(|error| io_error(source, error))? + .file_type() + .is_file() + { + return Err(io_error( + source, + io::Error::new(io::ErrorKind::InvalidData, "source is not a regular file"), + )); + } + Ok(input) +} + +fn io_error(path: &Path, source: io::Error) -> DumpShadowError { + DumpShadowError::Io { + path: path.to_path_buf(), + source, + } +} + +#[cfg(unix)] +fn sync_directory(path: &Path) -> Result<(), DumpShadowError> { + fs::File::open(path) + .and_then(|directory| directory.sync_all()) + .map_err(|source| io_error(path, source)) +} + +#[cfg(windows)] +fn sync_directory(path: &Path) -> Result<(), DumpShadowError> { + let directory = + crate::support::windows_fs::open_root(path).map_err(|source| io_error(path, source))?; + crate::support::windows_fs::flush(&directory).map_err(|source| io_error(path, source)) +} + +#[cfg(not(any(unix, windows)))] +fn sync_directory(_path: &Path) -> Result<(), DumpShadowError> { + Err(DumpShadowError::UnsupportedSafeFilesystem) +} + +#[cfg(test)] +mod tests { + use std::fs; + use std::path::{Path, PathBuf}; + + use sha2::{Digest, Sha256}; + use tempfile::tempdir; + + use super::{ + inspect_baseline, inspect_baseline_path, managed_manifest, publish_complete_baseline, + set_after_inventory_hook, set_before_baseline_seed_copy, BaselineInspection, DumpShadow, + EffectiveDumpMode, + }; + use crate::config::model::{BuilderBackend, InfobaseConfig, SourceFormat, SourceSetPurpose}; + use crate::domain::runtime_state::{ + BaselineRole, InfobaseIdentity, LogicalSourceRole, RuntimeSourceDescriptor, + RuntimeSourceIdentityInputs, RuntimeStateLayout, StateGeneration, + }; + use crate::domain::source_set::SourceSetContext; + use crate::use_cases::request::DumpModeRequest; + + fn context(base: &Path) -> SourceSetContext { + let source = base.join("source"); + fs::create_dir_all(&source).expect("source"); + let identity = InfobaseIdentity::normalize(&InfobaseConfig::file(format!( + "File={}", + base.join("ib").display() + ))) + .expect("identity"); + let layout = RuntimeStateLayout::new(base.join("work"), identity).expect("layout"); + let descriptor = RuntimeSourceDescriptor::new(RuntimeSourceIdentityInputs { + configured_source_identity: Path::new("source"), + source_root: &source, + purpose: SourceSetPurpose::Configuration, + format: SourceFormat::Designer, + backend: BuilderBackend::Designer, + logical_role: LogicalSourceRole::DesignerSource, + }) + .expect("descriptor"); + SourceSetContext::new("main", source, layout.source_state("main", &descriptor)) + } + + fn baseline( + context: &SourceSetContext, + generation: u64, + ) -> crate::domain::runtime_state::IbBaseline { + context.baseline( + BaselineRole::ConfiguredSource, + StateGeneration::new(generation), + ) + } + + fn valid_cdfi() -> &'static [u8] { + br#""# + } + + #[test] + fn baseline_round_trip_preserves_bytes_and_sorts_manifest() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + let dump = dir.path().join("dump"); + fs::create_dir_all(dump.join("z")).expect("nested"); + fs::write(dump.join("z/last.bin"), [0, 0xff, 7]).expect("binary"); + fs::write(dump.join("alpha.xml"), b"alpha\r\n").expect("alpha"); + fs::write(dump.join("CONFIGDUMPINFO.XML"), b"platform-owned").expect("cdfi"); + + let handle = baseline(&context, 1); + publish_complete_baseline(&dump, &[], &handle).expect("publish baseline"); + + let BaselineInspection::Valid(valid) = inspect_baseline(&handle).expect("inspect") else { + panic!("published baseline must be valid"); + }; + assert_eq!( + valid.files_root().join("alpha.xml").read_bytes(), + b"alpha\r\n" + ); + assert_eq!( + valid.files_root().join("z/last.bin").read_bytes(), + &[0, 0xff, 7] + ); + assert!(!valid.files_root().join("CONFIGDUMPINFO.XML").exists()); + + let manifest = fs::read_to_string(handle.path().join("manifest.json")).expect("manifest"); + assert!(manifest.starts_with("{\"version\":1,\"files\":[")); + assert!( + manifest.find("alpha.xml").expect("alpha") < manifest.find("z/last.bin").expect("z") + ); + assert_eq!(valid.files().len(), 2); + assert_eq!(valid.files()[0].path(), "alpha.xml"); + assert_eq!(valid.files()[0].byte_len(), 7); + let expected: [u8; 32] = Sha256::digest(b"alpha\r\n").into(); + assert_eq!(valid.files()[0].sha256(), expected); + } + + #[test] + fn managed_manifest_hashes_exact_bytes_and_excludes_private_entries() { + let dir = tempdir().expect("tempdir"); + let root = dir.path().join("source"); + let nested_work = root.join("build"); + fs::create_dir_all(root.join("nested")).expect("nested"); + fs::create_dir_all(&nested_work).expect("work"); + fs::write(root.join("alpha.xml"), b"alpha\r\n").expect("alpha"); + fs::write(root.join("nested/binary.bin"), [0, 0xff, 7]).expect("binary"); + fs::write(root.join("ConfigDumpInfo.xml"), b"platform-owned").expect("cdfi"); + fs::write(nested_work.join("ignored.xml"), b"ignored").expect("ignored"); + + let manifest = managed_manifest(&root, &[nested_work]).expect("manifest"); + + assert_eq!( + manifest.keys().cloned().collect::>(), + vec!["alpha.xml".to_owned(), "nested/binary.bin".to_owned(),] + ); + let alpha: [u8; 32] = Sha256::digest(b"alpha\r\n").into(); + let binary: [u8; 32] = Sha256::digest([0, 0xff, 7]).into(); + assert_eq!(manifest["alpha.xml"], alpha); + assert_eq!(manifest["nested/binary.bin"], binary); + } + + #[cfg(unix)] + #[test] + fn managed_manifest_does_not_follow_symlinks() { + use std::os::unix::fs::symlink; + + let dir = tempdir().expect("tempdir"); + let root = dir.path().join("source"); + let outside = dir.path().join("outside"); + fs::create_dir_all(&root).expect("source"); + fs::create_dir_all(&outside).expect("outside"); + fs::write(outside.join("secret.xml"), b"secret").expect("secret"); + symlink(&outside, root.join("linked-dir")).expect("linked dir"); + symlink(outside.join("secret.xml"), root.join("linked-file.xml")).expect("linked file"); + fs::write(root.join("managed.xml"), b"managed").expect("managed"); + + let manifest = managed_manifest(&root, &[]).expect("manifest"); + + assert_eq!( + manifest.keys().cloned().collect::>(), + vec!["managed.xml".to_owned()] + ); + } + + #[cfg(unix)] + #[test] + fn managed_manifest_rejects_parent_replaced_by_symlink_after_inventory() { + use std::os::unix::fs::symlink; + + let dir = tempdir().expect("tempdir"); + let root = dir.path().join("source"); + let outside = dir.path().join("outside"); + fs::create_dir_all(root.join("managed")).expect("managed"); + fs::create_dir(&outside).expect("outside"); + fs::write(root.join("managed/file.xml"), b"inside").expect("inside"); + fs::write(outside.join("file.xml"), b"outside").expect("outside file"); + let managed = root.join("managed"); + let displaced = root.join("managed-displaced"); + set_after_inventory_hook({ + let outside = outside.clone(); + move || { + fs::rename(&managed, displaced).expect("displace managed parent"); + symlink(outside, managed).expect("swap symlink"); + } + }); + + managed_manifest(&root, &[]).expect_err("parent symlink swap must fail"); + + assert_eq!(fs::read(outside.join("file.xml")).unwrap(), b"outside"); + } + + #[test] + fn manifest_observes_the_byte_exact_copied_version() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + let source_file = context.path().join("Configuration.xml"); + fs::write(&source_file, b"before").expect("source"); + set_after_inventory_hook({ + let source_file = source_file.clone(); + move || fs::write(source_file, b"after!").expect("replace after inventory") + }); + + let handle = baseline(&context, 1); + publish_complete_baseline(context.path(), &[], &handle).expect("baseline"); + + let BaselineInspection::Valid(valid) = inspect_baseline(&handle).expect("inspect") else { + panic!("copied baseline must remain self-consistent"); + }; + assert_eq!( + fs::read(valid.files_root().join("Configuration.xml")).expect("copied"), + b"after!" + ); + } + + #[test] + fn inspection_reports_missing_and_every_corrupt_baseline_shape() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + let source = context.path(); + fs::write(source.join("a.xml"), b"one").expect("source"); + + let missing = baseline(&context, 1); + assert!(matches!( + inspect_baseline(&missing).expect("missing"), + BaselineInspection::Missing + )); + + let corrupt_cases: &[(&str, fn(&Path))] = &[ + ("malformed manifest", |root| { + fs::write(root.join("manifest.json"), b"not-json").expect("manifest") + }), + ("missing file", |root| { + fs::remove_file(root.join("files/a.xml")).expect("remove") + }), + ("extra file", |root| { + fs::write(root.join("files/extra.xml"), b"extra").expect("extra") + }), + ("mismatched file", |root| { + fs::write(root.join("files/a.xml"), b"changed").expect("change") + }), + ]; + + for (index, (name, corrupt)) in corrupt_cases.iter().enumerate() { + let handle = baseline(&context, index as u64 + 2); + publish_complete_baseline(source, &[], &handle).expect("baseline"); + corrupt(handle.path()); + assert!( + matches!( + inspect_baseline(&handle).expect("inspect"), + BaselineInspection::Corrupt(_) + ), + "{name}" + ); + } + } + + #[cfg(unix)] + #[test] + fn inspection_rejects_symlink_and_nonregular_entries() { + use std::os::unix::fs::symlink; + + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + fs::write(context.path().join("a.xml"), b"one").expect("source"); + + let linked = baseline(&context, 1); + publish_complete_baseline(context.path(), &[], &linked).expect("baseline"); + fs::remove_file(linked.path().join("files/a.xml")).expect("remove"); + symlink( + context.path().join("a.xml"), + linked.path().join("files/a.xml"), + ) + .expect("link"); + assert!(matches!( + inspect_baseline(&linked).expect("inspect link"), + BaselineInspection::Corrupt(_) + )); + + let fifo = baseline(&context, 2); + publish_complete_baseline(context.path(), &[], &fifo).expect("baseline"); + fs::remove_file(fifo.path().join("files/a.xml")).expect("remove"); + let status = std::process::Command::new("mkfifo") + .arg(fifo.path().join("files/a.xml")) + .status() + .expect("mkfifo"); + assert!(status.success()); + assert!(matches!( + inspect_baseline(&fifo).expect("inspect fifo"), + BaselineInspection::Corrupt(_) + )); + } + + #[test] + fn incremental_shadow_seeds_valid_baseline_and_private_cdfi() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + fs::write(context.path().join("Configuration.xml"), b"baseline").expect("source"); + publish_complete_baseline(context.path(), &[], &baseline(&context, 1)).expect("baseline"); + fs::create_dir_all(context.private_cdfi_path().parent().expect("parent")).expect("state"); + fs::write(context.private_cdfi_path(), valid_cdfi()).expect("cdfi"); + + let shadow = DumpShadow::prepare( + &context, + BaselineRole::ConfiguredSource, + StateGeneration::new(1), + DumpModeRequest::Incremental, + ) + .expect("shadow"); + + assert_eq!(shadow.mode(), EffectiveDumpMode::Incremental); + assert_eq!( + fs::read(shadow.path().join("Configuration.xml")).expect("file"), + b"baseline" + ); + assert_eq!( + fs::read(shadow.path().join("ConfigDumpInfo.xml")).expect("cdfi"), + valid_cdfi() + ); + } + + #[test] + fn seed_promotes_to_full_when_baseline_changes_after_validation() { + for mutation in ["extra", "modified"] { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + fs::write(context.path().join("Configuration.xml"), b"baseline").expect("source"); + let handle = baseline(&context, 1); + publish_complete_baseline(context.path(), &[], &handle).expect("baseline"); + fs::create_dir_all(context.private_cdfi_path().parent().expect("parent")) + .expect("state"); + fs::write(context.private_cdfi_path(), valid_cdfi()).expect("cdfi"); + let files = handle.path().join("files"); + set_before_baseline_seed_copy(move || match mutation { + "extra" => fs::write(files.join("foreign.xml"), b"foreign").expect("extra"), + "modified" => { + fs::write(files.join("Configuration.xml"), b"changed").expect("modified") + } + _ => unreachable!(), + }); + + let shadow = DumpShadow::prepare( + &context, + BaselineRole::ConfiguredSource, + StateGeneration::new(1), + DumpModeRequest::Incremental, + ) + .expect("safe promotion"); + + assert_eq!(shadow.mode(), EffectiveDumpMode::Full); + assert!(fs::read_dir(shadow.path()) + .expect("empty shadow") + .next() + .is_none()); + } + } + + #[cfg(unix)] + #[test] + fn seed_parent_swap_promotes_to_full_without_reading_symlink_target() { + use std::os::unix::fs::symlink; + + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + fs::write(context.path().join("Configuration.xml"), b"baseline").expect("source"); + let handle = baseline(&context, 1); + publish_complete_baseline(context.path(), &[], &handle).expect("baseline"); + fs::create_dir_all(context.private_cdfi_path().parent().expect("parent")).expect("state"); + fs::write(context.private_cdfi_path(), valid_cdfi()).expect("cdfi"); + let files = handle.path().join("files"); + let displaced = handle.path().join("files-displaced"); + let outside = dir.path().join("outside"); + fs::create_dir(&outside).expect("outside"); + fs::write(outside.join("secret.xml"), b"secret").expect("secret"); + let outside_hook = outside.clone(); + set_before_baseline_seed_copy(move || { + fs::rename(&files, &displaced).expect("displace files"); + symlink(&outside_hook, &files).expect("swap files root"); + }); + + let shadow = DumpShadow::prepare( + &context, + BaselineRole::ConfiguredSource, + StateGeneration::new(1), + DumpModeRequest::Partial, + ) + .expect("safe promotion"); + + assert_eq!(shadow.mode(), EffectiveDumpMode::Full); + assert!(fs::read_dir(shadow.path()) + .expect("empty shadow") + .next() + .is_none()); + assert_eq!( + fs::read(outside.join("secret.xml")).expect("secret"), + b"secret" + ); + } + + #[test] + fn full_shadow_is_empty_even_when_seed_state_is_valid() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + fs::write(context.path().join("Configuration.xml"), b"baseline").expect("source"); + publish_complete_baseline(context.path(), &[], &baseline(&context, 1)).expect("baseline"); + fs::create_dir_all(context.private_cdfi_path().parent().expect("parent")).expect("state"); + fs::write(context.private_cdfi_path(), valid_cdfi()).expect("cdfi"); + + let shadow = DumpShadow::prepare( + &context, + BaselineRole::ConfiguredSource, + StateGeneration::new(1), + DumpModeRequest::Full, + ) + .expect("shadow"); + + assert_eq!(shadow.mode(), EffectiveDumpMode::Full); + assert!(fs::read_dir(shadow.path()) + .expect("shadow dir") + .next() + .is_none()); + } + + #[test] + fn missing_or_corrupt_seed_promotes_incremental_and_partial_to_full() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + fs::create_dir_all(context.private_cdfi_path().parent().expect("parent")).expect("state"); + fs::write(context.private_cdfi_path(), valid_cdfi()).expect("cdfi"); + + let missing = DumpShadow::prepare( + &context, + BaselineRole::ConfiguredSource, + StateGeneration::new(1), + DumpModeRequest::Incremental, + ) + .expect("missing baseline promotion"); + assert_eq!(missing.mode(), EffectiveDumpMode::Full); + + fs::write(context.path().join("Configuration.xml"), b"baseline").expect("source"); + publish_complete_baseline(context.path(), &[], &baseline(&context, 2)).expect("baseline"); + fs::write(context.private_cdfi_path(), b"broken xml").expect("corrupt cdfi"); + let corrupt = DumpShadow::prepare( + &context, + BaselineRole::ConfiguredSource, + StateGeneration::new(2), + DumpModeRequest::Partial, + ) + .expect("corrupt cdfi promotion"); + assert_eq!(corrupt.mode(), EffectiveDumpMode::Full); + assert!(fs::read_dir(corrupt.path()) + .expect("shadow dir") + .next() + .is_none()); + } + + #[test] + fn corrupt_baseline_or_missing_cdfi_also_promotes_to_full() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + fs::write(context.path().join("Configuration.xml"), b"baseline").expect("source"); + + let corrupt_handle = baseline(&context, 1); + publish_complete_baseline(context.path(), &[], &corrupt_handle).expect("baseline"); + fs::write(corrupt_handle.path().join("manifest.json"), b"broken").expect("corrupt"); + fs::create_dir_all(context.private_cdfi_path().parent().expect("parent")).expect("state"); + fs::write(context.private_cdfi_path(), valid_cdfi()).expect("cdfi"); + let corrupt_baseline = DumpShadow::prepare( + &context, + BaselineRole::ConfiguredSource, + StateGeneration::new(1), + DumpModeRequest::Incremental, + ) + .expect("corrupt baseline promotion"); + assert_eq!(corrupt_baseline.mode(), EffectiveDumpMode::Full); + + let valid_handle = baseline(&context, 2); + publish_complete_baseline(context.path(), &[], &valid_handle).expect("baseline"); + fs::remove_file(context.private_cdfi_path()).expect("remove cdfi"); + let missing_cdfi = DumpShadow::prepare( + &context, + BaselineRole::ConfiguredSource, + StateGeneration::new(2), + DumpModeRequest::Partial, + ) + .expect("missing cdfi promotion"); + assert_eq!(missing_cdfi.mode(), EffectiveDumpMode::Full); + } + + #[cfg(unix)] + #[test] + fn managed_manifest_accepts_symlinked_configured_root() { + use std::os::unix::fs::symlink; + + let dir = tempdir().expect("tempdir"); + let real = dir.path().join("real-source"); + fs::create_dir(&real).expect("real source"); + fs::write(real.join("Configuration.xml"), b"source").expect("source file"); + let linked = dir.path().join("configured-source"); + symlink(&real, &linked).expect("source symlink"); + + let manifest = managed_manifest(&linked, &[]).expect("manifest through symlink root"); + + assert!(manifest.contains_key("Configuration.xml")); + } + + #[test] + fn baseline_manifest_rejects_always_ignored_paths() { + let dir = tempdir().expect("tempdir"); + let baseline = dir.path().join("baseline"); + let files = baseline.join("files/.git"); + fs::create_dir_all(&files).expect("files"); + let bytes = b"foreign"; + fs::write(files.join("config"), bytes).expect("ignored file"); + let hash = format!("{:x}", Sha256::digest(bytes)); + fs::write( + baseline.join("manifest.json"), + serde_json::to_vec(&serde_json::json!({ + "version": 1, + "files": [{"path": ".git/config", "len": bytes.len(), "sha256": hash}] + })) + .expect("manifest json"), + ) + .expect("manifest"); + + let inspection = inspect_baseline_path(&baseline).expect("inspection"); + + assert!(matches!(inspection, BaselineInspection::Corrupt(_))); + } + + trait ReadBytes { + fn read_bytes(&self) -> Vec; + } + + impl ReadBytes for PathBuf { + fn read_bytes(&self) -> Vec { + fs::read(self).expect("read bytes") + } + } +} diff --git a/src/use_cases/mod.rs b/src/use_cases/mod.rs index 57e904c..d3cd081 100644 --- a/src/use_cases/mod.rs +++ b/src/use_cases/mod.rs @@ -20,6 +20,8 @@ pub mod context; pub mod convert_sources; /// Dump orchestration use case. pub mod dump_config; +mod dump_execution; +mod dump_shadow; /// Shared extension identity helpers. pub mod extension_identity; /// Shared discovery and preparation helpers for external artifacts. @@ -44,8 +46,12 @@ pub mod request; pub mod result; /// Test orchestration use case. pub mod run_tests; +mod runtime_state; +mod shadow_merge; /// Read-only source-set runtime indexes shared by orchestrating use cases. pub(crate) mod source_inventory; +mod source_publication; +mod source_transaction; /// Shared staged publication mechanics for full-replacement use-case outputs. mod staged_publication; /// Shared internal preparation for tool extensions. diff --git a/src/use_cases/result.rs b/src/use_cases/result.rs index 3ab7b86..1a8cd00 100644 --- a/src/use_cases/result.rs +++ b/src/use_cases/result.rs @@ -153,6 +153,7 @@ pub type UseCaseResult = Result>; mod tests { use super::{UseCaseError, UseCaseErrorKind}; use crate::config::loader::ConfigLoadError; + use crate::domain::runtime_state::RuntimeStateError; use crate::platform::designer::DesignerError; use crate::platform::edt_session::EdtSessionError; use crate::platform::ibcmd::IbcmdError; @@ -166,6 +167,24 @@ mod tests { assert_eq!(UseCaseErrorKind::Platform.exit_code(), 4); } + #[test] + fn runtime_state_config_errors_are_validation_and_path_errors_are_runtime() { + for error in [ + RuntimeStateError::EmptyConnection, + RuntimeStateError::MalformedConnectionString, + RuntimeStateError::MalformedRawConnection, + RuntimeStateError::UnsupportedRawConnection, + ] { + let validation = UseCaseError::from(AppError::from(error)); + assert_eq!(validation.kind(), UseCaseErrorKind::Validation); + } + let runtime = UseCaseError::from(AppError::from(RuntimeStateError::PathResolution( + std::io::Error::other("lookup failed"), + ))); + + assert_eq!(runtime.kind(), UseCaseErrorKind::Runtime); + } + #[test] fn process_app_error_normalizes_only_at_adapter_boundary() { let error = UseCaseError::from(AppError::PlatformProcess(ProcessError::SpawnFailed { diff --git a/src/use_cases/runtime_state.rs b/src/use_cases/runtime_state.rs new file mode 100644 index 0000000..88da205 --- /dev/null +++ b/src/use_cases/runtime_state.rs @@ -0,0 +1,3174 @@ +use std::collections::HashMap; +use std::fs; +use std::io::ErrorKind; +use std::io::{Read, Write}; +use std::path::{Path, PathBuf}; + +use quick_xml::events::{BytesStart, Event}; +use quick_xml::Reader; +use serde::{Deserialize, Serialize}; +use sha2::{Digest, Sha256}; +use thiserror::Error; +use uuid::Uuid; + +#[cfg(test)] +thread_local! { + static BEFORE_REDB_CLAIM_HOOK: std::cell::RefCell>> = + const { std::cell::RefCell::new(None) }; + static AFTER_REDB_CLAIM_HOOK: std::cell::RefCell>> = + const { std::cell::RefCell::new(None) }; + static BEFORE_REDB_ROLLBACK_HOOK: std::cell::RefCell>> = + const { std::cell::RefCell::new(None) }; + static FORCE_REDB_PUBLISH_FAILURE: std::cell::Cell = const { std::cell::Cell::new(false) }; + static DUMP_COMMIT_CRASH_PHASE: std::cell::Cell> = + const { std::cell::Cell::new(None) }; + static FAIL_DUMP_COMMIT_AFTER_REDB: std::cell::Cell = const { std::cell::Cell::new(false) }; + static BEFORE_BASELINE_DESTRUCTIVE_HOOK: std::cell::RefCell>> = + const { std::cell::RefCell::new(None) }; +} + +use crate::change_detection::analyzer::PreparedStateUpdate; +use crate::change_detection::hash_storage::{ + HashStorage, ObservedStorageState, StorageError, StoredFileState, +}; +use crate::domain::runtime_state::{BaselineRole, DumpTransactionId, StateGeneration}; +use crate::domain::source_set::SourceSetContext; +use crate::support::fs::{acquire_advisory_lock, AdvisoryLockGuard}; +use crate::use_cases::dump_shadow::{ + inspect_baseline_path, stage_complete_baseline, BaselineInspection, DumpShadowError, +}; + +pub(crate) struct DesignerStateLock { + path: PathBuf, + _guard: AdvisoryLockGuard, +} + +/// Complete private artifacts committed by a successful dump operation. +/// +/// The `SourceSetContext` passed to [`commit_dump_state_with_lock`] is the sole +/// owner of every artifact in this request. For EDT this is the configured EDT +/// context: its observation, configured-source baseline, optional intermediate +/// Designer baseline and private CDFI deliberately share one generation. The +/// generated Designer build context is not a second dump-state owner. +pub(crate) struct DumpStateCommitRequest<'a> { + prepared: &'a PreparedStateUpdate, + configured_source_root: &'a Path, + edt_platform_designer_root: Option<&'a Path>, + produced_cdfi: &'a Path, + transaction_id: DumpTransactionId, +} + +impl<'a> DumpStateCommitRequest<'a> { + pub(crate) fn new( + prepared: &'a PreparedStateUpdate, + configured_source_root: &'a Path, + produced_cdfi: &'a Path, + ) -> Self { + Self { + prepared, + configured_source_root, + edt_platform_designer_root: None, + produced_cdfi, + transaction_id: DumpTransactionId::new(), + } + } + + pub(crate) const fn with_edt_platform_designer(mut self, root: &'a Path) -> Self { + self.edt_platform_designer_root = Some(root); + self + } + + pub(crate) fn with_transaction_id(mut self, transaction_id: DumpTransactionId) -> Self { + self.transaction_id = transaction_id; + self + } +} + +#[cfg(test)] +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum DumpCommitCrashPhase { + AfterBaselines, + AfterCdfi, + AfterRedb, + AfterBaselineMarkerRemoval, +} + +#[cfg(test)] +fn set_dump_commit_crash_phase(phase: DumpCommitCrashPhase) { + DUMP_COMMIT_CRASH_PHASE.with(|slot| slot.set(Some(phase))); +} + +#[cfg(test)] +fn fail_next_dump_commit_after_redb() { + FAIL_DUMP_COMMIT_AFTER_REDB.with(|slot| slot.set(true)); +} + +#[cfg(test)] +fn inject_dump_crash(phase: DumpCommitCrashPhase) -> Result<(), RuntimeStateError> { + if DUMP_COMMIT_CRASH_PHASE.with(|slot| slot.get() == Some(phase)) { + DUMP_COMMIT_CRASH_PHASE.with(|slot| slot.set(None)); + Err(RuntimeStateError::InjectedDumpCrash(phase)) + } else { + Ok(()) + } +} + +pub(crate) fn lock_designer_state( + context: &SourceSetContext, +) -> Result { + let path = context.state_lock_path(); + let guard = acquire_advisory_lock(&path)?; + Ok(DesignerStateLock { + path, + _guard: guard, + }) +} + +fn validate_lock( + context: &SourceSetContext, + lock: &DesignerStateLock, +) -> Result<(), RuntimeStateError> { + if lock.path == context.state_lock_path() { + Ok(()) + } else { + Err(RuntimeStateError::InvalidJournal { + path: lock.path.clone(), + reason: "runtime-state lock belongs to another source context".to_owned(), + }) + } +} + +#[derive(Debug, Clone)] +pub(crate) struct ValidatedCdfi { + bytes: Vec, +} + +impl ValidatedCdfi { + pub(crate) fn bytes(&self) -> &[u8] { + &self.bytes + } +} + +#[derive(Debug, Clone)] +pub(crate) enum PrivateCdfiState { + Missing, + Valid(ValidatedCdfi), + Corrupt(String), +} + +#[derive(Debug, Error)] +pub(crate) enum RuntimeStateError { + #[error("failed to inspect private CDFI '{path}': {source}")] + Inspect { + path: PathBuf, + source: std::io::Error, + }, + #[error("failed to read private CDFI '{path}': {source}")] + Read { + path: PathBuf, + source: std::io::Error, + }, + #[error("invalid produced private CDFI '{path}': {reason}")] + InvalidProducedCdfi { path: PathBuf, reason: String }, + #[error("failed to prepare private runtime-state transaction: {0}")] + TransactionIo(#[from] std::io::Error), + #[error("failed to {operation} runtime-state path '{path}': {source}")] + Io { + operation: &'static str, + path: PathBuf, + source: std::io::Error, + }, + #[error("failed to prepare private hash storage: {0}")] + Storage(#[from] StorageError), + #[error( + "runtime state generation changed before publication: expected {expected}, found {actual}" + )] + StaleGeneration { expected: u64, actual: u64 }, + #[error("invalid runtime-state journal '{path}': {reason}")] + InvalidJournal { path: PathBuf, reason: String }, + #[error("runtime state generation cannot advance beyond u64::MAX")] + GenerationOverflow, + #[error("storage state changed before publication at '{path}'")] + StorageObservationChanged { path: PathBuf }, + #[error("failed to prepare private dump baseline: {0}")] + DumpShadow(Box), + #[cfg(test)] + #[error("injected dump-state crash after {0:?}")] + InjectedDumpCrash(DumpCommitCrashPhase), + #[error("runtime-state publication failed ({publication}); rollback also failed ({rollback}); journal retained at '{journal}'")] + PublicationAndRollback { + publication: Box, + rollback: Box, + journal: PathBuf, + }, +} + +impl From for RuntimeStateError { + fn from(error: DumpShadowError) -> Self { + Self::DumpShadow(Box::new(error)) + } +} + +#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)] +#[serde(rename_all = "snake_case")] +enum JournalStatus { + ClaimingRecoverable, + Prepared, + Committed, +} + +#[derive(Debug, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +struct StateJournal { + status: JournalStatus, + #[serde(default)] + generation: u64, + redb_existed: bool, + cdfi_existed: bool, + redb_staged: FileFingerprint, + cdfi_staged: FileFingerprint, + #[serde(default)] + baselines: Vec, + #[serde(default)] + dump_transaction_id: Option, +} + +#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, PartialOrd, Ord)] +#[serde(rename_all = "snake_case")] +enum JournalBaselineRole { + ConfiguredSource, + EdtPlatformDesigner, +} + +impl JournalBaselineRole { + const fn domain(self) -> BaselineRole { + match self { + Self::ConfiguredSource => BaselineRole::ConfiguredSource, + Self::EdtPlatformDesigner => BaselineRole::EdtPlatformDesigner, + } + } +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +struct JournalBaseline { + role: JournalBaselineRole, + staged_name: String, + ownership_token: String, + manifest_fingerprint: FileFingerprint, + #[serde(default)] + directory_identity: Option, +} + +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] +#[serde(deny_unknown_fields)] +struct FileFingerprint { + len: u64, + sha256: String, + identity: Option, +} + +impl FileFingerprint { + fn same_contents(&self, other: &Self) -> bool { + self.len == other.len && self.sha256 == other.sha256 + } +} + +#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)] +#[serde(deny_unknown_fields)] +struct FileIdentity { + volume: u64, + index: u64, +} + +const JOURNAL_FILE: &str = "journal.json"; +const STAGED_REDB: &str = "new-hash-storage.redb"; +const STAGED_CDFI: &str = "new-ConfigDumpInfo.xml"; +const BACKUP_REDB: &str = "old-hash-storage.redb"; +const BACKUP_CDFI: &str = "old-ConfigDumpInfo.xml"; +const BASELINE_OWNERSHIP_FILE: &str = ".runtime-state-transaction"; +const FULL_REBUILD_MARKER: &str = "full-rebuild-required"; + +pub(crate) fn designer_full_rebuild_required( + context: &SourceSetContext, +) -> Result { + full_rebuild_marker_path(context) + .try_exists() + .map_err(RuntimeStateError::TransactionIo) +} + +pub(crate) fn require_designer_full_rebuild( + context: &SourceSetContext, +) -> Result<(), RuntimeStateError> { + let marker = full_rebuild_marker_path(context); + if let Some(parent) = marker.parent() { + ensure_directory_synced(parent)?; + } + write_synced_file(&marker, b"previous Designer result was ambiguous\n") +} + +pub(crate) fn inspect_private_cdfi(path: &Path) -> Result { + let metadata = match fs::symlink_metadata(path) { + Ok(metadata) => metadata, + Err(source) if source.kind() == ErrorKind::NotFound => { + return Ok(PrivateCdfiState::Missing) + } + Err(source) => { + return Err(RuntimeStateError::Inspect { + path: path.to_path_buf(), + source, + }) + } + }; + if !metadata.file_type().is_file() || metadata.file_type().is_symlink() { + return Ok(PrivateCdfiState::Corrupt( + "private CDFI is not a regular file".to_owned(), + )); + } + let bytes = fs::read(path).map_err(|source| RuntimeStateError::Read { + path: path.to_path_buf(), + source, + })?; + match validate_cdfi_bytes(&bytes) { + Ok(()) => Ok(PrivateCdfiState::Valid(ValidatedCdfi { bytes })), + Err(reason) => Ok(PrivateCdfiState::Corrupt(reason)), + } +} + +fn validate_cdfi_bytes(bytes: &[u8]) -> Result<(), String> { + let mut reader = Reader::from_reader(bytes); + reader.config_mut().trim_text(true); + let mut root_seen = false; + let mut root_closed = false; + let mut depth = 0_u64; + let mut metadata_identity_seen = false; + loop { + match reader.read_event() { + Ok(Event::Start(event)) => { + if depth == 0 && root_seen { + return Err("CDFI contains more than one root element".to_owned()); + } + inspect_cdfi_element( + &event, + reader.decoder(), + &mut root_seen, + &mut metadata_identity_seen, + )?; + depth += 1; + } + Ok(Event::Empty(event)) => { + if depth == 0 && root_seen { + return Err("CDFI contains more than one root element".to_owned()); + } + inspect_cdfi_element( + &event, + reader.decoder(), + &mut root_seen, + &mut metadata_identity_seen, + )?; + if depth == 0 { + root_closed = true; + } + } + Ok(Event::End(_)) => { + if depth == 0 { + return Err("CDFI contains an unmatched closing element".to_owned()); + } + depth -= 1; + if depth == 0 { + root_closed = true; + } + } + Ok(Event::Text(text)) if depth == 0 => { + let decoded = text + .unescape() + .map_err(|error| format!("invalid text outside CDFI root: {error}"))?; + if !decoded.trim().is_empty() { + return Err("CDFI contains text outside the root element".to_owned()); + } + } + Ok(Event::CData(text)) if depth == 0 => { + let decoded = reader + .decoder() + .decode(text.as_ref()) + .map_err(|error| format!("invalid CDATA outside CDFI root: {error}"))?; + if !decoded.trim().is_empty() { + return Err("CDFI contains CDATA outside the root element".to_owned()); + } + } + Ok(Event::Eof) => break, + Ok(_) => {} + Err(error) => return Err(format!("malformed CDFI XML: {error}")), + } + } + if !root_seen || !root_closed || depth != 0 { + return Err("CDFI root element is missing or truncated".to_owned()); + } + if !metadata_identity_seen { + return Err("CDFI has no non-empty metadata identity/version".to_owned()); + } + Ok(()) +} + +fn inspect_cdfi_element( + event: &BytesStart<'_>, + decoder: quick_xml::encoding::Decoder, + root_seen: &mut bool, + metadata_identity_seen: &mut bool, +) -> Result<(), String> { + let local_name = event.local_name(); + if !*root_seen { + if local_name.as_ref() != b"ConfigDumpInfo" { + return Err("unexpected CDFI root element".to_owned()); + } + *root_seen = true; + let mut version = None; + for attribute in event.attributes() { + let attribute = attribute.map_err(|error| error.to_string())?; + if attribute.key.local_name().as_ref() == b"version" { + version = Some( + attribute + .decode_and_unescape_value(decoder) + .map_err(|error| error.to_string())? + .into_owned(), + ); + } + } + if version + .as_deref() + .is_none_or(|value| value.trim().is_empty()) + { + return Err("CDFI root has no non-empty version".to_owned()); + } + } else if local_name.as_ref() == b"Metadata" { + let mut id = None; + let mut config_version = None; + for attribute in event.attributes() { + let attribute = attribute.map_err(|error| error.to_string())?; + match attribute.key.local_name().as_ref() { + b"id" => { + id = Some( + attribute + .decode_and_unescape_value(decoder) + .map_err(|error| error.to_string())? + .into_owned(), + ) + } + b"configVersion" => { + config_version = Some( + attribute + .decode_and_unescape_value(decoder) + .map_err(|error| error.to_string())? + .into_owned(), + ) + } + _ => {} + } + } + if id.as_deref().is_some_and(|value| !value.trim().is_empty()) + && config_version + .as_deref() + .is_some_and(|value| !value.trim().is_empty()) + { + *metadata_identity_seen = true; + } + } + Ok(()) +} + +pub(crate) fn recover_designer_state(context: &SourceSetContext) -> Result<(), RuntimeStateError> { + let lock = lock_designer_state(context)?; + recover_designer_state_with_lock(context, &lock) +} + +pub(crate) fn recover_designer_state_with_lock( + context: &SourceSetContext, + lock: &DesignerStateLock, +) -> Result<(), RuntimeStateError> { + validate_lock(context, lock)?; + let transactions = context.transactions_dir(); + let entries = match fs::read_dir(&transactions) { + Ok(entries) => entries, + Err(error) if error.kind() == ErrorKind::NotFound => return Ok(()), + Err(error) => return Err(RuntimeStateError::TransactionIo(error)), + }; + for entry in entries { + let entry = entry?; + if !entry.file_type()?.is_dir() + || !entry.file_name().to_string_lossy().starts_with("state-") + { + continue; + } + recover_one_transaction(context, &entry.path())?; + } + Ok(()) +} + +#[cfg(test)] +pub(crate) fn commit_designer_state( + context: &SourceSetContext, + prepared: &PreparedStateUpdate, + produced_cdfi: &Path, +) -> Result<(), RuntimeStateError> { + let lock = lock_designer_state(context)?; + commit_designer_state_with_lock(context, &lock, prepared, produced_cdfi) +} + +pub(crate) fn commit_designer_state_with_lock( + context: &SourceSetContext, + lock: &DesignerStateLock, + prepared: &PreparedStateUpdate, + produced_cdfi: &Path, +) -> Result<(), RuntimeStateError> { + validate_lock(context, lock)?; + let produced = match inspect_private_cdfi(produced_cdfi)? { + PrivateCdfiState::Valid(cdfi) => cdfi, + PrivateCdfiState::Missing => { + require_designer_full_rebuild(context)?; + return Err(RuntimeStateError::InvalidProducedCdfi { + path: produced_cdfi.to_path_buf(), + reason: "platform did not produce ConfigDumpInfo.xml".to_owned(), + }); + } + PrivateCdfiState::Corrupt(reason) => { + require_designer_full_rebuild(context)?; + return Err(RuntimeStateError::InvalidProducedCdfi { + path: produced_cdfi.to_path_buf(), + reason, + }); + } + }; + recover_designer_state_with_lock(context, lock)?; + verify_storage_observation(context, &prepared.observed_storage)?; + + ensure_directory_synced(&context.transactions_dir())?; + let transaction = context + .transactions_dir() + .join(format!("state-{}", Uuid::new_v4())); + fs::create_dir(&transaction)?; + sync_directory(&context.transactions_dir())?; + let result = publish_state_transaction(context, prepared, &produced, &[], None, &transaction); + if let Err(publication) = result { + return match recover_one_transaction(context, &transaction) { + Ok(()) => Err(publication), + Err(rollback) => Err(RuntimeStateError::PublicationAndRollback { + publication: Box::new(publication), + rollback: Box::new(rollback), + journal: transaction, + }), + }; + } + result?; + clear_full_rebuild_marker(context) +} + +pub(crate) fn commit_dump_state_with_lock( + context: &SourceSetContext, + lock: &DesignerStateLock, + request: DumpStateCommitRequest<'_>, +) -> Result { + validate_lock(context, lock)?; + let produced = match inspect_private_cdfi(request.produced_cdfi)? { + PrivateCdfiState::Valid(cdfi) => cdfi, + PrivateCdfiState::Missing => { + return Err(RuntimeStateError::InvalidProducedCdfi { + path: request.produced_cdfi.to_path_buf(), + reason: "platform did not produce ConfigDumpInfo.xml".to_owned(), + }) + } + PrivateCdfiState::Corrupt(reason) => { + return Err(RuntimeStateError::InvalidProducedCdfi { + path: request.produced_cdfi.to_path_buf(), + reason, + }) + } + }; + recover_designer_state_with_lock(context, lock)?; + verify_storage_observation(context, &request.prepared.observed_storage)?; + let next_generation = request + .prepared + .observed_storage + .generation() + .checked_add(1) + .ok_or(RuntimeStateError::GenerationOverflow)?; + + ensure_directory_synced(&context.transactions_dir())?; + let transaction = context + .transactions_dir() + .join(format!("state-{}", Uuid::new_v4())); + fs::create_dir(&transaction)?; + sync_directory(&context.transactions_dir())?; + let mut baselines = vec![BaselineInput { + role: JournalBaselineRole::ConfiguredSource, + root: request.configured_source_root, + }]; + if let Some(root) = request.edt_platform_designer_root { + baselines.push(BaselineInput { + role: JournalBaselineRole::EdtPlatformDesigner, + root, + }); + } + let result = publish_state_transaction( + context, + request.prepared, + &produced, + &baselines, + Some(&request.transaction_id), + &transaction, + ); + if let Err(publication) = result { + #[cfg(test)] + if matches!(publication, RuntimeStateError::InjectedDumpCrash(_)) { + return Err(publication); + } + return match recover_one_transaction(context, &transaction) { + Ok(()) + if HashStorage::new(context.storage_path()) + .current_generation() + .is_ok_and(|generation| generation == next_generation) + && HashStorage::new(context.storage_path()) + .current_dump_transaction_id() + .is_ok_and(|transaction_id| { + transaction_id.as_ref() == Some(&request.transaction_id) + }) => + { + Ok(StateGeneration::new(next_generation)) + } + Ok(()) => Err(publication), + Err(rollback) => Err(RuntimeStateError::PublicationAndRollback { + publication: Box::new(publication), + rollback: Box::new(rollback), + journal: transaction, + }), + }; + } + result +} + +struct BaselineInput<'a> { + role: JournalBaselineRole, + root: &'a Path, +} + +pub(crate) fn cleanup_orphan_designer_transactions( + context: &SourceSetContext, + lock: &DesignerStateLock, +) -> Result<(), RuntimeStateError> { + validate_lock(context, lock)?; + let entries = match fs::read_dir(context.transactions_dir()) { + Ok(entries) => entries, + Err(error) if error.kind() == ErrorKind::NotFound => return Ok(()), + Err(error) => return Err(RuntimeStateError::TransactionIo(error)), + }; + for entry in entries { + let entry = entry?; + if entry.file_type()?.is_dir() + && entry + .file_name() + .to_string_lossy() + .starts_with("designer-build-") + { + fs::remove_dir_all(entry.path())?; + } + } + sync_directory(&context.transactions_dir()) +} + +fn clear_full_rebuild_marker(context: &SourceSetContext) -> Result<(), RuntimeStateError> { + let marker = full_rebuild_marker_path(context); + match fs::remove_file(&marker) { + Ok(()) => { + if let Some(parent) = marker.parent() { + sync_directory(parent)?; + } + Ok(()) + } + Err(error) if error.kind() == ErrorKind::NotFound => Ok(()), + Err(error) => Err(RuntimeStateError::TransactionIo(error)), + } +} + +fn full_rebuild_marker_path(context: &SourceSetContext) -> PathBuf { + context + .transactions_dir() + .parent() + .map(|state_dir| state_dir.join(FULL_REBUILD_MARKER)) + .unwrap_or_else(|| context.transactions_dir().join(FULL_REBUILD_MARKER)) +} + +fn verify_storage_observation( + context: &SourceSetContext, + expected: &ObservedStorageState, +) -> Result<(), RuntimeStateError> { + let storage = HashStorage::new(context.storage_path()); + match expected { + ObservedStorageState::MissingPath => match storage.load_state() { + Ok(crate::change_detection::hash_storage::HashStorageLoad::MissingPath) => Ok(()), + Ok(crate::change_detection::hash_storage::HashStorageLoad::ExistingUninitialized) + | Ok(crate::change_detection::hash_storage::HashStorageLoad::Initialized(_)) => { + Err(RuntimeStateError::StorageObservationChanged { + path: context.storage_path(), + }) + } + Err(error) => Err(RuntimeStateError::Storage(error)), + }, + ObservedStorageState::ExistingUninitialized { .. } => { + let actual = storage.uninitialized_observation()?; + if &actual == expected { + Ok(()) + } else { + Err(RuntimeStateError::StorageObservationChanged { + path: context.storage_path(), + }) + } + } + ObservedStorageState::Initialized { + generation: expected, + } => match storage.current_generation() { + Ok(actual) if actual == *expected => Ok(()), + Ok(actual) => Err(RuntimeStateError::StaleGeneration { + expected: *expected, + actual, + }), + Err(error) => Err(RuntimeStateError::Storage(error)), + }, + ObservedStorageState::Recoverable { .. } => { + let actual = storage.recoverable_observation()?; + if &actual == expected { + Ok(()) + } else { + Err(RuntimeStateError::StorageObservationChanged { + path: context.storage_path(), + }) + } + } + } +} + +fn publish_state_transaction( + context: &SourceSetContext, + prepared: &PreparedStateUpdate, + cdfi: &ValidatedCdfi, + baselines: &[BaselineInput<'_>], + dump_transaction_id: Option<&DumpTransactionId>, + transaction: &Path, +) -> Result { + let snapshot: HashMap = prepared + .snapshot + .iter() + .map(|file| { + ( + file.rel_path.clone(), + StoredFileState { + mtime_ns: file.mtime_ns, + hash: file.hash.clone(), + }, + ) + }) + .collect(); + let next_generation = prepared + .observed_storage + .generation() + .checked_add(1) + .ok_or(RuntimeStateError::GenerationOverflow)?; + if let Some(transaction_id) = dump_transaction_id { + HashStorage::create_dump_replacement( + transaction.join(STAGED_REDB), + &snapshot, + prepared.scan_started_at, + next_generation, + transaction_id, + )?; + } else { + HashStorage::create_replacement( + transaction.join(STAGED_REDB), + &snapshot, + prepared.scan_started_at, + next_generation, + )?; + } + write_synced_file(&transaction.join(STAGED_CDFI), cdfi.bytes())?; + let mut journal_baselines = Vec::with_capacity(baselines.len()); + for baseline in baselines { + let staged_name = match baseline.role { + JournalBaselineRole::ConfiguredSource => "new-baseline-configured-source", + JournalBaselineRole::EdtPlatformDesigner => "new-baseline-edt-platform-designer", + }; + let staged = transaction.join(staged_name); + stage_complete_baseline(baseline.root, &[], &staged)?; + let manifest_fingerprint = file_fingerprint(&staged.join("manifest.json"))?; + let ownership_token = Uuid::new_v4().to_string(); + write_synced_file( + &staged.join(BASELINE_OWNERSHIP_FILE), + ownership_token.as_bytes(), + )?; + journal_baselines.push(JournalBaseline { + role: baseline.role, + staged_name: staged_name.to_owned(), + ownership_token, + manifest_fingerprint, + directory_identity: opened_file_identity(&fs::File::open(&staged)?)?, + }); + } + let redb_staged = file_fingerprint(&transaction.join(STAGED_REDB))?; + let cdfi_staged = file_fingerprint(&transaction.join(STAGED_CDFI))?; + + let redb_target = context.storage_path(); + let cdfi_target = context.private_cdfi_path(); + let cdfi_existed = backup_regular_file(&cdfi_target, &transaction.join(BACKUP_CDFI))?; + if baselines.is_empty() && requires_storage_claim(&prepared.observed_storage) { + write_journal( + transaction, + &StateJournal { + status: JournalStatus::ClaimingRecoverable, + generation: next_generation, + redb_existed: true, + cdfi_existed, + redb_staged: redb_staged.clone(), + cdfi_staged: cdfi_staged.clone(), + baselines: journal_baselines.clone(), + dump_transaction_id: dump_transaction_id.cloned(), + }, + )?; + } + let redb_existed = if baselines.is_empty() { + prepare_redb_backup( + &redb_target, + &transaction.join(BACKUP_REDB), + &transaction.join(STAGED_REDB), + &prepared.observed_storage, + )? + } else { + let existed = backup_regular_file(&redb_target, &transaction.join(BACKUP_REDB))?; + verify_storage_observation(context, &prepared.observed_storage)?; + existed + }; + #[cfg(test)] + if baselines.is_empty() && requires_storage_claim(&prepared.observed_storage) { + AFTER_REDB_CLAIM_HOOK.with(|cell| { + if let Some(hook) = cell.borrow_mut().take() { + hook(); + } + }); + } + // Close the staging/backup TOCTOU window before the first live-file mutation. + if baselines.is_empty() && !requires_storage_claim(&prepared.observed_storage) { + verify_storage_observation(context, &prepared.observed_storage)?; + } + write_journal( + transaction, + &StateJournal { + status: JournalStatus::Prepared, + generation: next_generation, + redb_existed, + cdfi_existed, + redb_staged: redb_staged.clone(), + cdfi_staged: cdfi_staged.clone(), + baselines: journal_baselines.clone(), + dump_transaction_id: dump_transaction_id.cloned(), + }, + )?; + + for baseline in &journal_baselines { + publish_baseline( + context, + StateGeneration::new(next_generation), + transaction, + baseline, + )?; + } + #[cfg(test)] + if !baselines.is_empty() { + inject_dump_crash(DumpCommitCrashPhase::AfterBaselines)?; + } + publish_observed_file( + &transaction.join(STAGED_CDFI), + &cdfi_target, + &transaction.join(BACKUP_CDFI), + cdfi_existed, + )?; + #[cfg(test)] + if !baselines.is_empty() { + inject_dump_crash(DumpCommitCrashPhase::AfterCdfi)?; + } + if baselines.is_empty() && requires_storage_claim(&prepared.observed_storage) { + publish_absent_file(&transaction.join(STAGED_REDB), &redb_target)?; + } else { + // For dump commits redb stays at the old generation until this final CAS publication. + publish_observed_file( + &transaction.join(STAGED_REDB), + &redb_target, + &transaction.join(BACKUP_REDB), + redb_existed, + )?; + } + if let Some(state_dir) = redb_target.parent() { + sync_directory(state_dir)?; + } + #[cfg(test)] + if !baselines.is_empty() { + inject_dump_crash(DumpCommitCrashPhase::AfterRedb)?; + if FAIL_DUMP_COMMIT_AFTER_REDB.with(|slot| slot.replace(false)) { + return Err(RuntimeStateError::TransactionIo(std::io::Error::other( + "forced post-redb finalization failure", + ))); + } + } + remove_baseline_ownership_markers( + context, + StateGeneration::new(next_generation), + &journal_baselines, + )?; + write_journal( + transaction, + &StateJournal { + status: JournalStatus::Committed, + generation: next_generation, + redb_existed, + cdfi_existed, + redb_staged, + cdfi_staged, + baselines: journal_baselines, + dump_transaction_id: dump_transaction_id.cloned(), + }, + )?; + // Publication is committed at this point. Cleanup is deliberately best-effort; + // a surviving committed journal is removed by the next recovery pass. + fs::remove_dir_all(transaction)?; + if let Some(parent) = transaction.parent() { + sync_directory(parent)?; + } + Ok(StateGeneration::new(next_generation)) +} + +fn requires_storage_claim(observation: &ObservedStorageState) -> bool { + matches!( + observation, + ObservedStorageState::ExistingUninitialized { .. } + | ObservedStorageState::Initialized { .. } + | ObservedStorageState::Recoverable { .. } + ) +} + +fn publish_baseline( + context: &SourceSetContext, + generation: StateGeneration, + transaction: &Path, + baseline: &JournalBaseline, +) -> Result<(), RuntimeStateError> { + let staged = transaction.join(&baseline.staged_name); + let target = context.baseline(baseline.role.domain(), generation); + let parent = target + .path() + .parent() + .ok_or_else(|| RuntimeStateError::InvalidJournal { + path: target.path().to_path_buf(), + reason: "baseline target has no parent".to_owned(), + })?; + ensure_directory_synced(parent)?; + rename_directory_no_replace(&staged, target.path())?; + sync_rename_parents(&staged, target.path()) +} + +#[cfg(target_os = "macos")] +fn rename_directory_no_replace(source: &Path, target: &Path) -> Result<(), RuntimeStateError> { + rename_directory_with_flags(source, target, libc::RENAME_EXCL) +} + +#[cfg(target_os = "linux")] +fn rename_directory_no_replace(source: &Path, target: &Path) -> Result<(), RuntimeStateError> { + rename_directory_with_flags(source, target, libc::RENAME_NOREPLACE) +} + +#[cfg(any(target_os = "macos", target_os = "linux"))] +fn rename_directory_with_flags( + source: &Path, + target: &Path, + flags: libc::c_uint, +) -> Result<(), RuntimeStateError> { + use std::ffi::CString; + use std::os::unix::ffi::OsStrExt; + let source_c = CString::new(source.as_os_str().as_bytes()).map_err(|_| { + RuntimeStateError::InvalidJournal { + path: source.to_path_buf(), + reason: "staged baseline path contains NUL".to_owned(), + } + })?; + let target_c = CString::new(target.as_os_str().as_bytes()).map_err(|_| { + RuntimeStateError::InvalidJournal { + path: target.to_path_buf(), + reason: "baseline target path contains NUL".to_owned(), + } + })?; + #[cfg(target_os = "macos")] + // SAFETY: both values are NUL-free C strings valid for the duration of the syscall. + let result = unsafe { + libc::renameatx_np( + libc::AT_FDCWD, + source_c.as_ptr(), + libc::AT_FDCWD, + target_c.as_ptr(), + flags, + ) + }; + #[cfg(target_os = "linux")] + // SAFETY: both values are NUL-free C strings valid for the duration of the syscall. + let result = unsafe { + libc::renameat2( + libc::AT_FDCWD, + source_c.as_ptr(), + libc::AT_FDCWD, + target_c.as_ptr(), + flags, + ) + }; + if result == 0 { + Ok(()) + } else { + let source_error = std::io::Error::last_os_error(); + if matches!( + source_error.kind(), + ErrorKind::AlreadyExists | ErrorKind::DirectoryNotEmpty + ) { + Err(RuntimeStateError::StorageObservationChanged { + path: target.to_path_buf(), + }) + } else { + Err(runtime_io("publish staged baseline", target, source_error)) + } + } +} + +#[cfg(not(any(target_os = "macos", target_os = "linux")))] +fn rename_directory_no_replace(source: &Path, target: &Path) -> Result<(), RuntimeStateError> { + if target.try_exists()? { + return Err(RuntimeStateError::StorageObservationChanged { + path: target.to_path_buf(), + }); + } + fs::rename(source, target).map_err(|error| runtime_io("publish staged baseline", target, error)) +} + +enum BaselineOwnership { + Owned, + Missing, + Foreign, +} + +fn baseline_manifest_matches( + path: &Path, + expected: &FileFingerprint, +) -> Result { + match file_fingerprint(&path.join("manifest.json")) { + Ok(actual) => Ok(actual == *expected), + Err(RuntimeStateError::TransactionIo(error)) if error.kind() == ErrorKind::NotFound => { + Ok(false) + } + Err(RuntimeStateError::Io { source, .. }) if source.kind() == ErrorKind::NotFound => { + Ok(false) + } + Err(error) => Err(error), + } +} + +fn baseline_ownership( + path: &Path, + expected_token: &str, +) -> Result { + let marker = path.join(BASELINE_OWNERSHIP_FILE); + match fs::read(&marker) { + Ok(bytes) if bytes == expected_token.as_bytes() => Ok(BaselineOwnership::Owned), + Ok(_) => Ok(BaselineOwnership::Foreign), + Err(error) if error.kind() == ErrorKind::NotFound => Ok(BaselineOwnership::Missing), + Err(error) => Err(runtime_io("read baseline ownership marker", &marker, error)), + } +} + +fn baseline_tree_matches( + target: &Path, + baseline: &JournalBaseline, + marker_present: bool, +) -> Result { + let actual_identity = opened_file_identity(&fs::File::open(target)?)?; + if baseline.directory_identity.is_none() || actual_identity != baseline.directory_identity { + return Ok(false); + } + let mut root_entries = fs::read_dir(target)? + .map(|entry| entry.map(|entry| entry.file_name())) + .collect::, _>>()?; + root_entries.sort(); + let mut expected_entries = vec![ + std::ffi::OsString::from("files"), + std::ffi::OsString::from("manifest.json"), + ]; + if marker_present { + expected_entries.push(std::ffi::OsString::from(BASELINE_OWNERSHIP_FILE)); + expected_entries.sort(); + } + if root_entries != expected_entries + || !baseline_manifest_matches(target, &baseline.manifest_fingerprint)? + { + return Ok(false); + } + Ok(matches!( + inspect_baseline_path(target)?, + BaselineInspection::Valid(_) + )) +} + +fn baseline_directory_identity_matches( + target: &Path, + baseline: &JournalBaseline, +) -> Result { + let actual = opened_file_identity(&fs::File::open(target)?)?; + Ok(baseline.directory_identity.is_some() && actual == baseline.directory_identity) +} + +fn remove_baseline_ownership_markers( + context: &SourceSetContext, + generation: StateGeneration, + baselines: &[JournalBaseline], +) -> Result<(), RuntimeStateError> { + for baseline in baselines { + let target = context.baseline(baseline.role.domain(), generation); + finalize_baseline_marker(target.path(), baseline)?; + } + Ok(()) +} + +fn finalize_baseline_marker( + target: &Path, + baseline: &JournalBaseline, +) -> Result<(), RuntimeStateError> { + let claim = target.with_file_name(format!(".baseline-finalize-{}", baseline.ownership_token)); + let working = if claim.try_exists()? { + if target.try_exists()? { + return Err(RuntimeStateError::InvalidJournal { + path: target.to_path_buf(), + reason: "foreign baseline appeared while finalization was claimed".to_owned(), + }); + } + claim.as_path() + } else { + match baseline_ownership(target, &baseline.ownership_token)? { + BaselineOwnership::Missing + if target.is_dir() && baseline_tree_matches(target, baseline, false)? => + { + return Ok(()) + } + BaselineOwnership::Owned => { + rename_directory_no_replace(target, &claim)?; + claim.as_path() + } + BaselineOwnership::Missing | BaselineOwnership::Foreign => { + return Err(RuntimeStateError::InvalidJournal { + path: target.to_path_buf(), + reason: "published baseline ownership changed before commit".to_owned(), + }) + } + } + }; + match baseline_ownership(working, &baseline.ownership_token)? { + BaselineOwnership::Owned if baseline_tree_matches(working, baseline, true)? => { + fs::remove_file(working.join(BASELINE_OWNERSHIP_FILE))?; + sync_directory(working)?; + #[cfg(test)] + inject_dump_crash(DumpCommitCrashPhase::AfterBaselineMarkerRemoval)?; + } + BaselineOwnership::Missing if baseline_tree_matches(working, baseline, false)? => {} + BaselineOwnership::Owned | BaselineOwnership::Missing | BaselineOwnership::Foreign => { + if !target.try_exists()? { + rename_directory_no_replace(working, target)?; + } + return Err(RuntimeStateError::InvalidJournal { + path: target.to_path_buf(), + reason: "published baseline content or identity changed before commit".to_owned(), + }); + } + } + rename_directory_no_replace(working, target) +} + +fn rollback_baselines( + context: &SourceSetContext, + generation: StateGeneration, + baselines: &[JournalBaseline], +) -> Result<(), RuntimeStateError> { + for baseline in baselines { + let target = context.baseline(baseline.role.domain(), generation); + let claim = target + .path() + .with_file_name(format!(".baseline-rollback-{}", baseline.ownership_token)); + if claim.try_exists()? { + if !baseline_directory_identity_matches(&claim, baseline)? { + return Err(RuntimeStateError::InvalidJournal { + path: claim, + reason: "foreign claimed baseline prevents safe rollback".to_owned(), + }); + } + fs::remove_dir_all(&claim)?; + if let Some(parent) = claim.parent() { + sync_directory(parent)?; + } + continue; + } + match fs::symlink_metadata(target.path()) { + Ok(metadata) if metadata.file_type().is_dir() && !metadata.file_type().is_symlink() => { + match baseline_ownership(target.path(), &baseline.ownership_token)? { + BaselineOwnership::Owned + if baseline_tree_matches(target.path(), baseline, true)? => {} + BaselineOwnership::Owned => { + return Err(RuntimeStateError::InvalidJournal { + path: target.path().to_path_buf(), + reason: "foreign baseline content prevents safe rollback".to_owned(), + }) + } + BaselineOwnership::Missing | BaselineOwnership::Foreign => { + return Err(RuntimeStateError::InvalidJournal { + path: target.path().to_path_buf(), + reason: "foreign baseline prevents safe rollback".to_owned(), + }) + } + } + rename_directory_no_replace(target.path(), &claim)?; + if !baseline_tree_matches(&claim, baseline, true)? { + rename_directory_no_replace(&claim, target.path())?; + return Err(RuntimeStateError::InvalidJournal { + path: target.path().to_path_buf(), + reason: "foreign baseline content prevents safe rollback".to_owned(), + }); + } + #[cfg(test)] + BEFORE_BASELINE_DESTRUCTIVE_HOOK.with(|slot| { + if let Some(hook) = slot.borrow_mut().take() { + hook(); + } + }); + fs::remove_dir_all(&claim)?; + if let Some(parent) = claim.parent() { + sync_directory(parent)?; + } + } + Ok(_) => { + return Err(RuntimeStateError::InvalidJournal { + path: target.path().to_path_buf(), + reason: "foreign non-directory baseline prevents safe rollback".to_owned(), + }) + } + Err(error) if error.kind() == ErrorKind::NotFound => {} + Err(error) => return Err(RuntimeStateError::TransactionIo(error)), + } + } + Ok(()) +} + +fn prepare_redb_backup( + target: &Path, + backup: &Path, + staged: &Path, + observation: &ObservedStorageState, +) -> Result { + if requires_storage_claim(observation) { + backup_regular_file(target, backup)?; + #[cfg(test)] + BEFORE_REDB_CLAIM_HOOK.with(|cell| { + if let Some(hook) = cell.borrow_mut().take() { + hook(); + } + }); + let original_claim = transaction_original_claim_path(staged)?; + fs::rename(target, &original_claim) + .map_err(|error| runtime_io("claim observed hash storage", target, error))?; + sync_rename_parents(target, &original_claim)?; + if !file_fingerprint(backup)?.same_contents(&file_fingerprint(&original_claim)?) { + restore_claimed_original(&original_claim, target)?; + return Err(RuntimeStateError::StorageObservationChanged { + path: target.to_path_buf(), + }); + } + let claimed_storage = HashStorage::new(original_claim.clone()); + let claimed = match observation { + ObservedStorageState::ExistingUninitialized { .. } => { + claimed_storage.uninitialized_observation()? + } + ObservedStorageState::Recoverable { .. } => { + claimed_storage.recoverable_observation()? + } + ObservedStorageState::Initialized { generation } => { + match claimed_storage.current_generation() { + Ok(actual) if actual == *generation => { + ObservedStorageState::Initialized { generation: actual } + } + Ok(_) => { + let failure = RuntimeStateError::StorageObservationChanged { + path: target.to_path_buf(), + }; + return Err(restore_claimed_original_after_error( + &original_claim, + target, + failure, + )); + } + Err(error) => { + return Err(restore_claimed_original_after_error( + &original_claim, + target, + RuntimeStateError::Storage(error), + )); + } + } + } + ObservedStorageState::MissingPath => { + restore_claimed_original(&original_claim, target)?; + return Err(RuntimeStateError::StorageObservationChanged { + path: target.to_path_buf(), + }); + } + }; + if &claimed == observation { + fs::File::open(&original_claim) + .and_then(|file| file.sync_all()) + .map_err(|error| runtime_io("sync claimed hash storage", &original_claim, error))?; + return Ok(true); + } + restore_claimed_original(&original_claim, target)?; + return Err(RuntimeStateError::StorageObservationChanged { + path: target.to_path_buf(), + }); + } + backup_regular_file(target, backup) +} + +fn restore_claimed_original(claim: &Path, target: &Path) -> Result<(), RuntimeStateError> { + publish_absent_file(claim, target) +} + +fn restore_claimed_original_after_error( + claim: &Path, + target: &Path, + failure: RuntimeStateError, +) -> RuntimeStateError { + match restore_claimed_original(claim, target) { + Ok(()) => failure, + Err(rollback) => RuntimeStateError::PublicationAndRollback { + publication: Box::new(failure), + rollback: Box::new(rollback), + journal: claim.to_path_buf(), + }, + } +} + +fn backup_regular_file(target: &Path, backup: &Path) -> Result { + match fs::symlink_metadata(target) { + Ok(metadata) if metadata.file_type().is_file() && !metadata.file_type().is_symlink() => { + fs::copy(target, backup)?; + fs::File::open(backup)?.sync_all()?; + if let Some(parent) = backup.parent() { + sync_directory(parent)?; + } + Ok(true) + } + Ok(_) => Err(RuntimeStateError::InvalidJournal { + path: target.to_path_buf(), + reason: "runtime-state target is not a regular file".to_owned(), + }), + Err(error) if error.kind() == ErrorKind::NotFound => Ok(false), + Err(error) => Err(RuntimeStateError::TransactionIo(error)), + } +} + +fn write_synced_file(path: &Path, bytes: &[u8]) -> Result<(), RuntimeStateError> { + let mut file = + fs::File::create(path).map_err(|error| runtime_io("create synced file", path, error))?; + file.write_all(bytes) + .map_err(|error| runtime_io("write synced file", path, error))?; + file.sync_all() + .map_err(|error| runtime_io("sync file", path, error))?; + if let Some(parent) = path.parent() { + sync_directory(parent)?; + } + Ok(()) +} + +fn ensure_directory_synced(path: &Path) -> Result<(), RuntimeStateError> { + let existed = path.try_exists()?; + fs::create_dir_all(path)?; + if !existed { + if let Some(parent) = path.parent() { + sync_directory(parent)?; + } + } + Ok(()) +} + +fn publish_absent_file(staged: &Path, target: &Path) -> Result<(), RuntimeStateError> { + if let Some(parent) = target.parent() { + fs::create_dir_all(parent)?; + } + #[cfg(test)] + if FORCE_REDB_PUBLISH_FAILURE.with(|forced| forced.replace(false)) { + return Err(runtime_io( + "publish staged file without overwriting", + target, + std::io::Error::other("forced redb publication failure"), + )); + } + fs::hard_link(staged, target).map_err(|error| { + if error.kind() == ErrorKind::AlreadyExists { + RuntimeStateError::StorageObservationChanged { + path: target.to_path_buf(), + } + } else { + runtime_io("publish staged file without overwriting", target, error) + } + })?; + if staged.parent() != target.parent() { + if let Some(parent) = staged.parent() { + sync_directory(parent)?; + } + } + if let Some(parent) = target.parent() { + sync_directory(parent)?; + } + Ok(()) +} + +fn publish_observed_file( + staged: &Path, + target: &Path, + backup: &Path, + existed: bool, +) -> Result<(), RuntimeStateError> { + if !existed { + return publish_absent_file(staged, target); + } + let claimed_current = transaction_original_claim_path(staged)?; + fs::rename(target, &claimed_current) + .map_err(|error| runtime_io("claim observed live file for publication", target, error))?; + sync_rename_parents(target, &claimed_current)?; + if fs::read(&claimed_current)? != fs::read(backup)? { + match publish_absent_file(&claimed_current, target) { + Ok(()) | Err(RuntimeStateError::StorageObservationChanged { .. }) => {} + Err(error) => return Err(error), + } + return Err(RuntimeStateError::StorageObservationChanged { + path: target.to_path_buf(), + }); + } + match publish_absent_file(staged, target) { + Ok(()) => Ok(()), + Err(publication) => { + let restoration = publish_absent_file(&claimed_current, target); + match restoration { + Ok(()) | Err(RuntimeStateError::StorageObservationChanged { .. }) => { + Err(publication) + } + Err(rollback) => Err(RuntimeStateError::PublicationAndRollback { + publication: Box::new(publication), + rollback: Box::new(rollback), + journal: claimed_current, + }), + } + } + } +} + +fn sync_rename_parents(source: &Path, destination: &Path) -> Result<(), RuntimeStateError> { + if let Some(parent) = destination.parent() { + sync_directory(parent)?; + } + if destination.parent() != source.parent() { + if let Some(parent) = source.parent() { + sync_directory(parent)?; + } + } + Ok(()) +} + +fn runtime_io(operation: &'static str, path: &Path, source: std::io::Error) -> RuntimeStateError { + RuntimeStateError::Io { + operation, + path: path.to_path_buf(), + source, + } +} + +fn write_journal(transaction: &Path, journal: &StateJournal) -> Result<(), RuntimeStateError> { + let path = transaction.join(JOURNAL_FILE); + let temporary = transaction.join("journal.tmp"); + let bytes = serde_json::to_vec(journal).map_err(|error| RuntimeStateError::InvalidJournal { + path: path.clone(), + reason: error.to_string(), + })?; + let mut file = fs::File::create(&temporary)?; + file.write_all(&bytes)?; + file.sync_all()?; + fs::rename(temporary, path)?; + sync_directory(transaction)?; + Ok(()) +} + +fn validate_state_journal( + journal: &StateJournal, + journal_path: &Path, +) -> Result<(), RuntimeStateError> { + let invalid = |reason: &str| RuntimeStateError::InvalidJournal { + path: journal_path.to_path_buf(), + reason: reason.to_owned(), + }; + if journal.generation == 0 { + return Err(invalid("generation must be nonzero")); + } + for (label, fingerprint) in [ + ("redb", &journal.redb_staged), + ("cdfi", &journal.cdfi_staged), + ] { + validate_journal_fingerprint(fingerprint) + .map_err(|reason| invalid(&format!("invalid {label} fingerprint: {reason}")))?; + } + if journal.status == JournalStatus::ClaimingRecoverable && !journal.baselines.is_empty() { + return Err(invalid( + "recoverable-claim journal cannot contain dump baselines", + )); + } + if journal.baselines.is_empty() != journal.dump_transaction_id.is_none() { + return Err(invalid( + "dump transaction id must exist exactly for journals with dump baselines", + )); + } + let mut roles = std::collections::BTreeSet::new(); + for baseline in &journal.baselines { + if !roles.insert(baseline.role) { + return Err(invalid("baseline roles must be unique")); + } + let expected_name = match baseline.role { + JournalBaselineRole::ConfiguredSource => "new-baseline-configured-source", + JournalBaselineRole::EdtPlatformDesigner => "new-baseline-edt-platform-designer", + }; + if baseline.staged_name != expected_name { + return Err(invalid("baseline staged name does not match its role")); + } + let mut components = Path::new(&baseline.ownership_token).components(); + if !matches!(components.next(), Some(std::path::Component::Normal(_))) + || components.next().is_some() + || !Uuid::parse_str(&baseline.ownership_token) + .is_ok_and(|uuid| uuid.hyphenated().to_string() == baseline.ownership_token) + { + return Err(invalid( + "baseline ownership token must be one canonical UUID component", + )); + } + validate_journal_fingerprint(&baseline.manifest_fingerprint).map_err(|reason| { + invalid(&format!("invalid baseline manifest fingerprint: {reason}")) + })?; + } + Ok(()) +} + +fn validate_journal_fingerprint(fingerprint: &FileFingerprint) -> Result<(), &'static str> { + if fingerprint.sha256.len() != 64 + || !fingerprint + .sha256 + .bytes() + .all(|byte| byte.is_ascii_digit() || matches!(byte, b'a'..=b'f')) + { + return Err("SHA-256 must be 64 lowercase hexadecimal characters"); + } + Ok(()) +} + +fn recover_one_transaction( + context: &SourceSetContext, + transaction: &Path, +) -> Result<(), RuntimeStateError> { + let journal_path = transaction.join(JOURNAL_FILE); + let bytes = match fs::read(&journal_path) { + Ok(bytes) => bytes, + Err(error) if error.kind() == ErrorKind::NotFound => { + fs::remove_dir_all(transaction)?; + if let Some(parent) = transaction.parent() { + sync_directory(parent)?; + } + return Ok(()); + } + Err(error) => return Err(RuntimeStateError::TransactionIo(error)), + }; + let journal: StateJournal = + serde_json::from_slice(&bytes).map_err(|error| RuntimeStateError::InvalidJournal { + path: journal_path.clone(), + reason: error.to_string(), + })?; + validate_state_journal(&journal, &journal_path)?; + let generation = StateGeneration::new(journal.generation); + let matching_generation_visible = if journal.generation == 0 { + false + } else { + recovery_observation_matches( + HashStorage::new(context.storage_path()).current_generation(), + |actual| actual == journal.generation, + )? + }; + let matching_dump_transaction = match &journal.dump_transaction_id { + Some(expected) => recovery_observation_matches( + HashStorage::new(context.storage_path()).current_dump_transaction_id(), + |actual| actual.as_ref() == Some(expected), + )?, + None => true, + }; + let generation_visible = matching_generation_visible + && matching_dump_transaction + && same_file_identity(&context.storage_path(), &transaction.join(STAGED_REDB))?; + if matching_generation_visible && !generation_visible { + return Err(RuntimeStateError::InvalidJournal { + path: context.storage_path(), + reason: "foreign hash storage uses the pending generation".to_owned(), + }); + } + match journal.status { + JournalStatus::ClaimingRecoverable => restore_transaction_target( + &context.storage_path(), + &transaction.join(BACKUP_REDB), + &transaction.join(STAGED_REDB), + true, + &journal.redb_staged, + )?, + JournalStatus::Prepared if generation_visible => { + if !same_file_identity(&context.private_cdfi_path(), &transaction.join(STAGED_CDFI))? { + return Err(RuntimeStateError::InvalidJournal { + path: context.private_cdfi_path(), + reason: "foreign CDFI prevents visible-generation recovery".to_owned(), + }); + } + remove_baseline_ownership_markers(context, generation, &journal.baselines)?; + } + JournalStatus::Prepared => { + #[cfg(test)] + BEFORE_REDB_ROLLBACK_HOOK.with(|cell| { + if let Some(hook) = cell.borrow_mut().take() { + hook(); + } + }); + restore_transaction_target( + &context.storage_path(), + &transaction.join(BACKUP_REDB), + &transaction.join(STAGED_REDB), + journal.redb_existed, + &journal.redb_staged, + )?; + restore_transaction_target( + &context.private_cdfi_path(), + &transaction.join(BACKUP_CDFI), + &transaction.join(STAGED_CDFI), + journal.cdfi_existed, + &journal.cdfi_staged, + )?; + rollback_baselines(context, generation, &journal.baselines)?; + } + JournalStatus::Committed => { + for baseline in &journal.baselines { + let target = context.baseline(baseline.role.domain(), generation); + finalize_baseline_marker(target.path(), baseline)?; + } + } + } + fs::remove_dir_all(transaction)?; + if let Some(parent) = transaction.parent() { + sync_directory(parent)?; + } + Ok(()) +} + +fn recovery_observation_matches( + observation: Result, + predicate: impl FnOnce(T) -> bool, +) -> Result { + match observation { + Ok(value) => Ok(predicate(value)), + Err(error) if error.is_recoverable() => Ok(false), + Err(error) => Err(RuntimeStateError::Storage(error)), + } +} + +fn restore_transaction_target( + target: &Path, + backup: &Path, + staged: &Path, + existed: bool, + staged_fingerprint: &FileFingerprint, +) -> Result<(), RuntimeStateError> { + let original_claim = transaction_original_claim_path(staged)?; + let published_claim = transaction_published_claim_path(staged)?; + if target.try_exists()? { + if published_claim.try_exists()? { + return Ok(()); + } + fs::rename(target, &published_claim).map_err(|error| { + runtime_io("claim live file for rollback classification", target, error) + })?; + sync_rename_parents(target, &published_claim)?; + } + if published_claim.try_exists()? { + let staged_is_valid = + staged.try_exists()? && file_fingerprint(staged)? == *staged_fingerprint; + let published_is_ours = staged_is_valid && same_file_identity(&published_claim, staged)?; + if !published_is_ours { + match publish_absent_file(&published_claim, target) { + Ok(()) | Err(RuntimeStateError::StorageObservationChanged { .. }) => return Ok(()), + Err(error) => return Err(error), + } + } + } + if existed && !target.try_exists()? { + let backup_exists = backup.try_exists()?; + let original_exists = original_claim.try_exists()?; + let restore_source = if original_exists { + original_claim.as_path() + } else if backup_exists { + backup + } else { + return Err(RuntimeStateError::InvalidJournal { + path: backup.to_path_buf(), + reason: "runtime-state backup is missing during rollback".to_owned(), + }); + }; + publish_absent_file(restore_source, target)?; + } + Ok(()) +} + +fn transaction_original_claim_path(staged: &Path) -> Result { + transaction_artifact_path(staged, ".original.claimed") +} + +fn transaction_published_claim_path(staged: &Path) -> Result { + transaction_artifact_path(staged, ".published.claimed") +} + +fn transaction_artifact_path(staged: &Path, suffix: &str) -> Result { + let file_name = staged + .file_name() + .ok_or_else(|| RuntimeStateError::InvalidJournal { + path: staged.to_path_buf(), + reason: "staged runtime-state path has no file name".to_owned(), + })?; + let mut claimed_name = file_name.to_os_string(); + claimed_name.push(suffix); + Ok(staged.with_file_name(claimed_name)) +} + +fn file_fingerprint(path: &Path) -> Result { + let metadata = fs::symlink_metadata(path)?; + if !metadata.file_type().is_file() || metadata.file_type().is_symlink() { + return Err(RuntimeStateError::InvalidJournal { + path: path.to_path_buf(), + reason: "runtime-state fingerprint target is not a regular file".to_owned(), + }); + } + let mut file = fs::File::open(path)?; + let mut digest = Sha256::new(); + let mut buffer = [0_u8; 64 * 1024]; + let mut len = 0_u64; + loop { + let read = file.read(&mut buffer)?; + if read == 0 { + break; + } + digest.update(&buffer[..read]); + len = len + .checked_add(read as u64) + .ok_or(RuntimeStateError::InvalidJournal { + path: path.to_path_buf(), + reason: "runtime-state file length overflowed u64".to_owned(), + })?; + } + Ok(FileFingerprint { + len, + sha256: format!("{:x}", digest.finalize()), + identity: opened_file_identity(&file)?, + }) +} + +fn same_file_identity(left: &Path, right: &Path) -> Result { + let left_file = fs::File::open(left)?; + let right_file = fs::File::open(right)?; + let left_identity = opened_file_identity(&left_file)?; + let right_identity = opened_file_identity(&right_file)?; + Ok(left_identity.is_some() && left_identity == right_identity) +} + +#[cfg(unix)] +fn opened_file_identity(file: &fs::File) -> Result, RuntimeStateError> { + use std::os::unix::fs::MetadataExt; + let metadata = file.metadata()?; + Ok(Some(FileIdentity { + volume: metadata.dev(), + index: metadata.ino(), + })) +} + +#[cfg(windows)] +fn opened_file_identity(file: &fs::File) -> Result, RuntimeStateError> { + windows_file_identity(file).map(Some) +} + +#[cfg(not(any(unix, windows)))] +fn opened_file_identity(_file: &fs::File) -> Result, RuntimeStateError> { + Ok(None) +} + +#[cfg(windows)] +fn windows_file_identity(file: &fs::File) -> Result { + use std::mem::MaybeUninit; + use std::os::windows::io::AsRawHandle; + use windows_sys::Win32::Storage::FileSystem::{ + GetFileInformationByHandle, BY_HANDLE_FILE_INFORMATION, + }; + let mut information = MaybeUninit::::uninit(); + // SAFETY: the handle comes from a live File and the API initializes the + // output structure on success. + let succeeded = unsafe { + GetFileInformationByHandle(file.as_raw_handle().cast(), information.as_mut_ptr()) + }; + if succeeded == 0 { + return Err(RuntimeStateError::TransactionIo( + std::io::Error::last_os_error(), + )); + } + // SAFETY: guarded by the successful Win32 call above. + let information = unsafe { information.assume_init() }; + Ok(FileIdentity { + volume: u64::from(information.dwVolumeSerialNumber), + index: (u64::from(information.nFileIndexHigh) << 32) | u64::from(information.nFileIndexLow), + }) +} + +#[cfg(unix)] +fn sync_directory(path: &Path) -> Result<(), RuntimeStateError> { + fs::File::open(path)?.sync_all()?; + Ok(()) +} + +#[cfg(not(unix))] +fn sync_directory(_path: &Path) -> Result<(), RuntimeStateError> { + // Rust does not expose a portable directory-fsync primitive. File contents are + // still synced, and the persisted journal remains available for recovery. + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::{ + commit_designer_state, commit_dump_state_with_lock, fail_next_dump_commit_after_redb, + file_fingerprint, inspect_private_cdfi, lock_designer_state, prepare_redb_backup, + recover_designer_state, set_dump_commit_crash_phase, transaction_original_claim_path, + DumpCommitCrashPhase, DumpStateCommitRequest, JournalStatus, PrivateCdfiState, + RuntimeStateError, StateJournal, AFTER_REDB_CLAIM_HOOK, BACKUP_CDFI, BACKUP_REDB, + BASELINE_OWNERSHIP_FILE, BEFORE_BASELINE_DESTRUCTIVE_HOOK, BEFORE_REDB_CLAIM_HOOK, + BEFORE_REDB_ROLLBACK_HOOK, FORCE_REDB_PUBLISH_FAILURE, JOURNAL_FILE, STAGED_CDFI, + STAGED_REDB, + }; + use crate::change_detection::analyzer::{PreparedFileState, PreparedStateUpdate}; + use crate::change_detection::hash_storage::ObservedStorageState; + use crate::change_detection::hash_storage::{HashStorage, StoredFileState}; + use crate::config::model::{BuilderBackend, InfobaseConfig, SourceFormat, SourceSetPurpose}; + use crate::domain::runtime_state::{ + BaselineRole, DumpTransactionId, InfobaseIdentity, LogicalSourceRole, + RuntimeSourceDescriptor, RuntimeSourceIdentityInputs, RuntimeStateLayout, StateGeneration, + }; + use crate::domain::source_set::SourceSetContext; + use std::collections::HashMap; + use std::fs; + use std::path::Path; + use std::sync::mpsc; + use std::time::Duration; + use tempfile::tempdir; + use uuid::Uuid; + + const VALID: &str = r#""#; + + fn prepared_journal( + transaction: &Path, + redb_existed: bool, + cdfi_existed: bool, + ) -> StateJournal { + StateJournal { + status: JournalStatus::Prepared, + generation: 99, + redb_existed, + cdfi_existed, + redb_staged: file_fingerprint(&transaction.join(STAGED_REDB)) + .expect("staged redb fingerprint"), + cdfi_staged: file_fingerprint(&transaction.join(STAGED_CDFI)) + .expect("staged CDFI fingerprint"), + baselines: Vec::new(), + dump_transaction_id: None, + } + } + + #[test] + fn classifies_valid_missing_and_corrupt_private_cdfi() { + let dir = tempdir().expect("tempdir"); + let path = dir.path().join("ConfigDumpInfo.xml"); + assert!(matches!( + inspect_private_cdfi(&path).expect("missing"), + PrivateCdfiState::Missing + )); + + fs::write(&path, VALID).expect("valid"); + assert!(matches!( + inspect_private_cdfi(&path).expect("valid"), + PrivateCdfiState::Valid(_) + )); + + fs::write(&path, "").expect("invalid"); + assert!(matches!( + inspect_private_cdfi(&path).expect("corrupt"), + PrivateCdfiState::Corrupt(_) + )); + } + + #[test] + fn rejects_missing_platform_owned_identity_values() { + let dir = tempdir().expect("tempdir"); + let path = dir.path().join("ConfigDumpInfo.xml"); + fs::write( + &path, + r#""#, + ) + .expect("invalid cdfi"); + assert!(matches!( + inspect_private_cdfi(&path).expect("classification"), + PrivateCdfiState::Corrupt(_) + )); + } + + #[test] + fn rejects_truncated_cdfi_with_valid_identity_prefix() { + let dir = tempdir().expect("tempdir"); + let path = dir.path().join("ConfigDumpInfo.xml"); + fs::write( + &path, + r#""#, + ) + .expect("truncated CDFI"); + + assert!(matches!( + inspect_private_cdfi(&path).expect("classification"), + PrivateCdfiState::Corrupt(_) + )); + } + + #[test] + fn rejects_non_whitespace_text_or_cdata_outside_root() { + let dir = tempdir().expect("tempdir"); + let path = dir.path().join("ConfigDumpInfo.xml"); + for xml in [ + format!("prefix{VALID}"), + format!("{VALID}suffix"), + format!("{VALID}"), + ] { + fs::write(&path, xml).expect("invalid CDFI"); + assert!(matches!( + inspect_private_cdfi(&path).expect("classification"), + PrivateCdfiState::Corrupt(_) + )); + } + } + + #[test] + fn rejects_whitespace_only_decoded_identity_attributes() { + let dir = tempdir().expect("tempdir"); + let path = dir.path().join("ConfigDumpInfo.xml"); + for xml in [ + r#""#, + r#""#, + r#""#, + ] { + fs::write(&path, xml).expect("invalid CDFI"); + assert!(matches!( + inspect_private_cdfi(&path).expect("classification"), + PrivateCdfiState::Corrupt(_) + )); + } + } + + #[test] + fn accepts_bom_and_namespaced_cdfi_fixture() { + let dir = tempdir().expect("tempdir"); + let path = dir.path().join("ConfigDumpInfo.xml"); + fs::write( + &path, + b"\xEF\xBB\xBF", + ) + .expect("fixture"); + + assert!(matches!( + inspect_private_cdfi(&path).expect("fixture state"), + PrivateCdfiState::Valid(_) + )); + } + + #[test] + fn accepts_repository_designer_cdfi_fixture() { + let dir = tempdir().expect("tempdir"); + let path = dir.path().join("ConfigDumpInfo.xml"); + fs::write( + &path, + include_bytes!("../../tests/fixtures/designer/configuration/ConfigDumpInfo.xml"), + ) + .expect("fixture"); + + assert!(matches!( + inspect_private_cdfi(&path).expect("fixture state"), + PrivateCdfiState::Valid(_) + )); + } + + #[cfg(unix)] + #[test] + fn symlink_and_directory_are_corrupt_private_state() { + use std::os::unix::fs::symlink; + + let dir = tempdir().expect("tempdir"); + let target = dir.path().join("target.xml"); + fs::write(&target, VALID).expect("target"); + let symlink_path = dir.path().join("ConfigDumpInfo.xml"); + symlink(&target, &symlink_path).expect("symlink"); + assert!(matches!( + inspect_private_cdfi(&symlink_path).expect("symlink state"), + PrivateCdfiState::Corrupt(_) + )); + + fs::remove_file(&symlink_path).expect("remove symlink"); + fs::create_dir(&symlink_path).expect("directory"); + assert!(matches!( + inspect_private_cdfi(&symlink_path).expect("directory state"), + PrivateCdfiState::Corrupt(_) + )); + } + + #[cfg(unix)] + #[test] + fn surfaces_private_cdfi_permission_errors_as_hard_failures() { + use std::os::unix::fs::PermissionsExt; + + let dir = tempdir().expect("tempdir"); + let path = dir.path().join("ConfigDumpInfo.xml"); + fs::write(&path, VALID).expect("CDFI"); + fs::set_permissions(&path, fs::Permissions::from_mode(0o000)).expect("deny read"); + let result = inspect_private_cdfi(&path); + fs::set_permissions(&path, fs::Permissions::from_mode(0o600)).expect("restore read"); + + assert!(result.is_err()); + } + + fn context(root: &Path) -> SourceSetContext { + let source = root.join("source"); + fs::create_dir_all(&source).expect("source"); + let identity = InfobaseIdentity::normalize(&InfobaseConfig::file(format!( + "File={}", + root.join("ib").display() + ))) + .expect("identity"); + let layout = RuntimeStateLayout::new(root.join("work"), identity).expect("layout"); + let descriptor = RuntimeSourceDescriptor::new(RuntimeSourceIdentityInputs { + configured_source_identity: Path::new("source"), + source_root: &source, + purpose: SourceSetPurpose::Configuration, + format: SourceFormat::Designer, + backend: BuilderBackend::Designer, + logical_role: LogicalSourceRole::DesignerSource, + }) + .expect("descriptor"); + SourceSetContext::new("main", source, layout.source_state("main", &descriptor)) + } + + fn seed_state(context: &SourceSetContext, cdfi: &str) { + let mut snapshot = HashMap::new(); + snapshot.insert( + "Configuration.xml".to_owned(), + StoredFileState { + mtime_ns: 1, + hash: "a".repeat(64), + }, + ); + HashStorage::new(context.storage_path()) + .commit_snapshot(&snapshot, 1, 0) + .expect("seed storage"); + fs::write(context.private_cdfi_path(), cdfi).expect("seed CDFI"); + } + + fn prepared(observed_generation: u64) -> PreparedStateUpdate { + PreparedStateUpdate { + snapshot: vec![PreparedFileState { + rel_path: "Configuration.xml".to_owned(), + mtime_ns: 2, + hash: "b".repeat(64), + }], + scan_started_at: 2, + observed_storage: ObservedStorageState::Initialized { + generation: observed_generation, + }, + } + } + + fn dump_roots(root: &Path) -> (std::path::PathBuf, std::path::PathBuf) { + let configured = root.join("configured-shadow"); + let edt = root.join("edt-platform-shadow"); + fs::create_dir_all(&configured).expect("configured shadow"); + fs::create_dir_all(&edt).expect("EDT platform shadow"); + fs::write(configured.join("Configuration.xml"), b"configured-v2").expect("configured file"); + fs::write(configured.join("ConfigDumpInfo.xml"), b"must be excluded") + .expect("configured CDFI sentinel"); + fs::write(edt.join("Configuration.xml"), b"edt-platform-v2").expect("EDT file"); + (configured, edt) + } + + fn malicious_baseline_journal( + transaction: &Path, + staged_name: String, + ownership_token: String, + ) -> StateJournal { + fs::write(transaction.join(STAGED_REDB), b"staged-redb").expect("staged redb"); + fs::write(transaction.join(STAGED_CDFI), b"staged-cdfi").expect("staged cdfi"); + let mut journal = prepared_journal(transaction, false, false); + journal.generation = 2; + journal.baselines.push(super::JournalBaseline { + role: super::JournalBaselineRole::ConfiguredSource, + staged_name, + ownership_token, + manifest_fingerprint: file_fingerprint(&transaction.join(STAGED_CDFI)) + .expect("fingerprint"), + directory_identity: None, + }); + journal + } + + #[test] + fn recovery_rejects_unbounded_baseline_journal_fields_before_mutation() { + for (staged_name, ownership_token) in [ + ( + "/tmp/v8-runner-external-baseline".to_owned(), + Uuid::new_v4().to_string(), + ), + ( + "../external-baseline".to_owned(), + Uuid::new_v4().to_string(), + ), + ( + "new-baseline-configured-source".to_owned(), + "../external-token".to_owned(), + ), + ] { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + let transaction = context.transactions_dir().join("state-malicious"); + fs::create_dir_all(&transaction).expect("transaction"); + let sentinel = dir.path().join("external-sentinel"); + fs::write(&sentinel, b"preserve").expect("sentinel"); + let journal = malicious_baseline_journal(&transaction, staged_name, ownership_token); + fs::write( + transaction.join(JOURNAL_FILE), + serde_json::to_vec(&journal).expect("journal"), + ) + .expect("write journal"); + + recover_designer_state(&context).expect_err("invalid journal must fail closed"); + + assert_eq!(fs::read(&sentinel).expect("sentinel"), b"preserve"); + assert!(transaction.exists()); + } + } + + #[test] + fn recovery_rejects_unknown_journal_fields_before_mutation() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + let transaction = context.transactions_dir().join("state-unknown-field"); + fs::create_dir_all(&transaction).expect("transaction"); + fs::write(transaction.join(STAGED_REDB), b"staged-redb").expect("staged redb"); + fs::write(transaction.join(STAGED_CDFI), b"staged-cdfi").expect("staged cdfi"); + let mut value = serde_json::to_value(prepared_journal(&transaction, false, false)) + .expect("journal value"); + value + .as_object_mut() + .expect("journal object") + .insert("unknown".to_owned(), serde_json::json!(true)); + fs::write( + transaction.join(JOURNAL_FILE), + serde_json::to_vec(&value).expect("journal"), + ) + .expect("write journal"); + + recover_designer_state(&context).expect_err("unknown field must fail closed"); + + assert!(transaction.exists()); + } + + #[test] + fn dump_commit_publishes_both_baselines_cdfi_and_observation_as_one_generation() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + seed_state(&context, VALID); + let produced = dir.path().join("produced.xml"); + let next = r#""#; + fs::write(&produced, next).expect("produced CDFI"); + let (configured, edt) = dump_roots(dir.path()); + let lock = lock_designer_state(&context).expect("lock"); + let transaction_id = DumpTransactionId::new(); + + let generation = commit_dump_state_with_lock( + &context, + &lock, + DumpStateCommitRequest::new(&prepared(1), &configured, &produced) + .with_edt_platform_designer(&edt) + .with_transaction_id(transaction_id.clone()), + ) + .expect("dump state commit"); + + assert_eq!(generation, StateGeneration::new(2)); + assert_eq!( + HashStorage::new(context.storage_path()) + .current_dump_transaction_id() + .expect("dump transaction"), + Some(transaction_id) + ); + assert_eq!( + HashStorage::new(context.storage_path()) + .current_generation() + .expect("generation"), + 2 + ); + assert_eq!( + fs::read_to_string(context.private_cdfi_path()).unwrap(), + next + ); + for role in [ + BaselineRole::ConfiguredSource, + BaselineRole::EdtPlatformDesigner, + ] { + let baseline = context.baseline(role, generation); + assert!(baseline.path().join("manifest.json").is_file()); + assert!(!baseline.path().join("files/ConfigDumpInfo.xml").exists()); + } + } + + #[test] + fn dump_commit_returns_visible_generation_after_forward_recovery() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + seed_state(&context, VALID); + let produced = dir.path().join("produced.xml"); + fs::write( + &produced, + r#""#, + ) + .expect("produced CDFI"); + let (configured, edt) = dump_roots(dir.path()); + let lock = lock_designer_state(&context).expect("lock"); + fail_next_dump_commit_after_redb(); + + let generation = commit_dump_state_with_lock( + &context, + &lock, + DumpStateCommitRequest::new(&prepared(1), &configured, &produced) + .with_edt_platform_designer(&edt), + ) + .expect("forward recovery made the next generation coherent"); + + assert_eq!(generation, StateGeneration::new(2)); + assert_eq!( + HashStorage::new(context.storage_path()) + .current_generation() + .expect("visible generation"), + 2 + ); + } + + #[test] + fn dump_commit_recovery_never_exposes_a_mixed_generation() { + for phase in [ + DumpCommitCrashPhase::AfterBaselines, + DumpCommitCrashPhase::AfterCdfi, + DumpCommitCrashPhase::AfterRedb, + ] { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + seed_state(&context, VALID); + let produced = dir.path().join("produced.xml"); + fs::write( + &produced, + r#""#, + ) + .expect("produced CDFI"); + let (configured, edt) = dump_roots(dir.path()); + let lock = lock_designer_state(&context).expect("lock"); + set_dump_commit_crash_phase(phase); + + commit_dump_state_with_lock( + &context, + &lock, + DumpStateCommitRequest::new(&prepared(1), &configured, &produced) + .with_edt_platform_designer(&edt), + ) + .expect_err("injected crash"); + drop(lock); + recover_designer_state(&context).expect("restart recovery"); + + let observed = HashStorage::new(context.storage_path()) + .current_generation() + .expect("generation after recovery"); + let next = StateGeneration::new(2); + if phase == DumpCommitCrashPhase::AfterRedb { + assert_eq!(observed, 2); + assert!(context + .baseline(BaselineRole::ConfiguredSource, next) + .path() + .is_dir()); + assert!(context + .baseline(BaselineRole::EdtPlatformDesigner, next) + .path() + .is_dir()); + assert!(fs::read_to_string(context.private_cdfi_path()) + .expect("new CDFI") + .contains("id=\"crash\"")); + } else { + assert_eq!(observed, 1); + assert!(!context + .baseline(BaselineRole::ConfiguredSource, next) + .path() + .exists()); + assert!(!context + .baseline(BaselineRole::EdtPlatformDesigner, next) + .path() + .exists()); + assert_eq!( + fs::read_to_string(context.private_cdfi_path()).unwrap(), + VALID + ); + } + } + } + + #[test] + fn dump_commit_redb_failure_rolls_back_all_previously_published_artifacts() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + seed_state(&context, VALID); + let produced = dir.path().join("produced.xml"); + fs::write( + &produced, + r#""#, + ) + .expect("produced CDFI"); + let (configured, edt) = dump_roots(dir.path()); + let lock = lock_designer_state(&context).expect("lock"); + FORCE_REDB_PUBLISH_FAILURE.with(|forced| forced.set(true)); + + commit_dump_state_with_lock( + &context, + &lock, + DumpStateCommitRequest::new(&prepared(1), &configured, &produced) + .with_edt_platform_designer(&edt), + ) + .expect_err("forced redb failure"); + + assert_eq!( + HashStorage::new(context.storage_path()) + .current_generation() + .expect("old generation"), + 1 + ); + assert_eq!( + fs::read_to_string(context.private_cdfi_path()).unwrap(), + VALID + ); + let next = StateGeneration::new(2); + assert!(!context + .baseline(BaselineRole::ConfiguredSource, next) + .path() + .exists()); + assert!(!context + .baseline(BaselineRole::EdtPlatformDesigner, next) + .path() + .exists()); + } + + #[test] + fn recovery_never_deletes_a_foreign_baseline() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + seed_state(&context, VALID); + let produced = dir.path().join("produced.xml"); + fs::write( + &produced, + r#""#, + ) + .expect("produced CDFI"); + let (configured, _) = dump_roots(dir.path()); + let lock = lock_designer_state(&context).expect("lock"); + set_dump_commit_crash_phase(DumpCommitCrashPhase::AfterBaselines); + commit_dump_state_with_lock( + &context, + &lock, + DumpStateCommitRequest::new(&prepared(1), &configured, &produced), + ) + .expect_err("injected crash"); + drop(lock); + let baseline = context.baseline(BaselineRole::ConfiguredSource, StateGeneration::new(2)); + fs::write( + baseline.path().join(BASELINE_OWNERSHIP_FILE), + b"foreign-owner", + ) + .expect("replace ownership marker"); + + recover_designer_state(&context).expect_err("foreign baseline blocks rollback"); + + assert!(baseline.path().is_dir()); + assert_eq!( + fs::read(baseline.path().join(BASELINE_OWNERSHIP_FILE)).unwrap(), + b"foreign-owner" + ); + assert_eq!( + HashStorage::new(context.storage_path()) + .current_generation() + .expect("old generation"), + 1 + ); + } + + #[test] + fn recovery_never_deletes_foreign_content_added_to_an_owned_baseline() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + seed_state(&context, VALID); + let produced = dir.path().join("produced.xml"); + fs::write(&produced, VALID).expect("produced CDFI"); + let (configured, _) = dump_roots(dir.path()); + let lock = lock_designer_state(&context).expect("lock"); + set_dump_commit_crash_phase(DumpCommitCrashPhase::AfterBaselines); + commit_dump_state_with_lock( + &context, + &lock, + DumpStateCommitRequest::new(&prepared(1), &configured, &produced), + ) + .expect_err("injected crash"); + drop(lock); + let baseline = context.baseline(BaselineRole::ConfiguredSource, StateGeneration::new(2)); + fs::write(baseline.path().join("foreign.txt"), b"foreign").expect("foreign file"); + + recover_designer_state(&context).expect_err("foreign baseline content blocks rollback"); + + assert_eq!( + fs::read(baseline.path().join("foreign.txt")).unwrap(), + b"foreign" + ); + assert!(baseline.path().is_dir()); + } + + #[test] + fn recovery_never_deletes_an_owned_baseline_with_modified_managed_content() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + seed_state(&context, VALID); + let produced = dir.path().join("produced.xml"); + fs::write(&produced, VALID).expect("produced CDFI"); + let (configured, _) = dump_roots(dir.path()); + let lock = lock_designer_state(&context).expect("lock"); + set_dump_commit_crash_phase(DumpCommitCrashPhase::AfterBaselines); + commit_dump_state_with_lock( + &context, + &lock, + DumpStateCommitRequest::new(&prepared(1), &configured, &produced), + ) + .expect_err("injected crash"); + drop(lock); + let baseline = context.baseline(BaselineRole::ConfiguredSource, StateGeneration::new(2)); + let managed = baseline.path().join("files/Configuration.xml"); + fs::write(&managed, b"foreign").expect("modify managed baseline file"); + + recover_designer_state(&context).expect_err("modified baseline blocks rollback"); + + assert_eq!(fs::read(managed).unwrap(), b"foreign"); + assert!(baseline.path().is_dir()); + } + + #[test] + fn recovery_never_deletes_a_replaced_baseline_with_a_copied_marker() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + seed_state(&context, VALID); + let produced = dir.path().join("produced.xml"); + fs::write(&produced, VALID).expect("produced CDFI"); + let (configured, _) = dump_roots(dir.path()); + let lock = lock_designer_state(&context).expect("lock"); + set_dump_commit_crash_phase(DumpCommitCrashPhase::AfterBaselines); + commit_dump_state_with_lock( + &context, + &lock, + DumpStateCommitRequest::new(&prepared(1), &configured, &produced), + ) + .expect_err("injected crash"); + drop(lock); + let baseline = context.baseline(BaselineRole::ConfiguredSource, StateGeneration::new(2)); + let displaced = baseline.path().with_extension("displaced"); + fs::rename(baseline.path(), &displaced).expect("displace owned baseline"); + fs::create_dir_all(baseline.path().join("files")).expect("replacement baseline"); + for relative in [ + "manifest.json", + "files/Configuration.xml", + BASELINE_OWNERSHIP_FILE, + ] { + fs::copy(displaced.join(relative), baseline.path().join(relative)) + .expect("copy baseline artifact"); + } + + recover_designer_state(&context).expect_err("replaced baseline blocks rollback"); + + assert!(baseline.path().is_dir()); + assert!(baseline.path().join("files/Configuration.xml").is_file()); + } + + #[test] + fn rollback_claims_owned_baseline_before_a_foreign_replacement_can_appear() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + seed_state(&context, VALID); + let produced = dir.path().join("produced.xml"); + fs::write(&produced, VALID).expect("produced CDFI"); + let (configured, _) = dump_roots(dir.path()); + let lock = lock_designer_state(&context).expect("lock"); + set_dump_commit_crash_phase(DumpCommitCrashPhase::AfterBaselines); + commit_dump_state_with_lock( + &context, + &lock, + DumpStateCommitRequest::new(&prepared(1), &configured, &produced), + ) + .expect_err("injected crash"); + drop(lock); + let baseline = context.baseline(BaselineRole::ConfiguredSource, StateGeneration::new(2)); + let target = baseline.path().to_path_buf(); + let displaced = target.with_extension("hook-displaced"); + BEFORE_BASELINE_DESTRUCTIVE_HOOK.with(|slot| { + *slot.borrow_mut() = Some(Box::new(move || { + if target.exists() { + fs::rename(&target, displaced).expect("displace checked baseline"); + } + fs::create_dir_all(&target).expect("foreign replacement"); + fs::write(target.join("foreign.txt"), b"foreign").expect("foreign content"); + })); + }); + + recover_designer_state(&context).expect("owned claim rollback"); + + assert_eq!( + fs::read(baseline.path().join("foreign.txt")).unwrap(), + b"foreign" + ); + } + + #[test] + fn recovery_finishes_baseline_marker_removal_after_crash() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + seed_state(&context, VALID); + let produced = dir.path().join("produced.xml"); + fs::write(&produced, VALID).expect("produced CDFI"); + let (configured, _) = dump_roots(dir.path()); + let lock = lock_designer_state(&context).expect("lock"); + set_dump_commit_crash_phase(DumpCommitCrashPhase::AfterBaselineMarkerRemoval); + + commit_dump_state_with_lock( + &context, + &lock, + DumpStateCommitRequest::new(&prepared(1), &configured, &produced), + ) + .expect_err("injected marker-removal crash"); + drop(lock); + recover_designer_state(&context).expect("restart recovery"); + + let baseline = context.baseline(BaselineRole::ConfiguredSource, StateGeneration::new(2)); + assert!(baseline.path().is_dir()); + assert!(!baseline.path().join(BASELINE_OWNERSHIP_FILE).exists()); + } + + #[test] + fn recovery_finishes_partially_removed_owned_rollback_claim() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + seed_state(&context, VALID); + let produced = dir.path().join("produced.xml"); + fs::write(&produced, VALID).expect("produced CDFI"); + let (configured, _) = dump_roots(dir.path()); + let lock = lock_designer_state(&context).expect("lock"); + set_dump_commit_crash_phase(DumpCommitCrashPhase::AfterBaselines); + commit_dump_state_with_lock( + &context, + &lock, + DumpStateCommitRequest::new(&prepared(1), &configured, &produced), + ) + .expect_err("injected baseline crash"); + drop(lock); + let baseline = context.baseline(BaselineRole::ConfiguredSource, StateGeneration::new(2)); + let token = String::from_utf8( + fs::read(baseline.path().join(BASELINE_OWNERSHIP_FILE)).expect("ownership token"), + ) + .expect("utf8 token"); + let claim = baseline + .path() + .with_file_name(format!(".baseline-rollback-{token}")); + fs::rename(baseline.path(), &claim).expect("persist rollback claim"); + fs::remove_file(claim.join("files/Configuration.xml")).expect("partial cleanup"); + + recover_designer_state(&context).expect("resume partial rollback cleanup"); + + assert!(!claim.exists()); + assert!(!baseline.path().exists()); + } + + #[test] + fn dump_commit_does_not_replace_an_empty_foreign_baseline_directory() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + seed_state(&context, VALID); + let produced = dir.path().join("produced.xml"); + fs::write(&produced, VALID).expect("produced CDFI"); + let (configured, _) = dump_roots(dir.path()); + let foreign = context.baseline(BaselineRole::ConfiguredSource, StateGeneration::new(2)); + fs::create_dir_all(foreign.path()).expect("empty foreign baseline"); + let lock = lock_designer_state(&context).expect("lock"); + + commit_dump_state_with_lock( + &context, + &lock, + DumpStateCommitRequest::new(&prepared(1), &configured, &produced), + ) + .expect_err("foreign target must reject publication"); + + assert!(foreign.path().is_dir()); + assert_eq!(fs::read_dir(foreign.path()).unwrap().count(), 0); + assert_eq!( + HashStorage::new(context.storage_path()) + .current_generation() + .expect("old generation"), + 1 + ); + } + + #[test] + fn successful_commit_publishes_cdfi_and_hash_generation_together() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + seed_state(&context, VALID); + let produced = dir.path().join("produced.xml"); + let next = r#""#; + fs::write(&produced, next).expect("produced"); + + commit_designer_state(&context, &prepared(1), &produced).expect("commit"); + + assert_eq!( + HashStorage::new(context.storage_path()) + .load_snapshot() + .expect("snapshot") + .generation, + 2 + ); + assert_eq!( + fs::read_to_string(context.private_cdfi_path()).expect("CDFI"), + next + ); + } + + #[test] + fn recoverable_claim_preserves_concurrently_repaired_storage() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + let storage_path = context.storage_path(); + fs::create_dir_all(storage_path.parent().expect("state dir")).expect("state dir"); + fs::write(&storage_path, b"corrupt-a").expect("corrupt A"); + fs::write(context.private_cdfi_path(), VALID).expect("private CDFI"); + let observed = HashStorage::new(storage_path.clone()) + .recoverable_observation() + .expect("recoverable token"); + + let healthy_path = dir.path().join("healthy.redb"); + let mut snapshot = HashMap::new(); + snapshot.insert( + "Configuration.xml".to_owned(), + StoredFileState { + mtime_ns: 7, + hash: "c".repeat(64), + }, + ); + HashStorage::create_replacement(healthy_path.clone(), &snapshot, 7, 9).expect("healthy B"); + let healthy_bytes = fs::read(&healthy_path).expect("healthy bytes"); + let target_for_hook = storage_path.clone(); + BEFORE_REDB_CLAIM_HOOK.with(|cell| { + *cell.borrow_mut() = Some(Box::new(move || { + fs::remove_file(&target_for_hook).expect("remove corrupt A"); + fs::rename(&healthy_path, &target_for_hook).expect("publish healthy B"); + })); + }); + let produced = dir.path().join("produced.xml"); + fs::write(&produced, VALID).expect("produced CDFI"); + let mut update = prepared(0); + update.observed_storage = observed; + + commit_designer_state(&context, &update, &produced) + .expect_err("changed storage must reject publication"); + assert_eq!( + fs::read(&storage_path).expect("healthy storage retained"), + healthy_bytes + ); + assert_eq!( + HashStorage::new(storage_path) + .load_snapshot() + .expect("healthy storage") + .generation, + 9 + ); + } + + #[test] + fn initialized_claim_preserves_concurrent_in_place_commit() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + seed_state(&context, VALID); + let target = context.storage_path(); + BEFORE_REDB_CLAIM_HOOK.with(|cell| { + *cell.borrow_mut() = Some(Box::new(move || { + HashStorage::new(target) + .commit_snapshot(&HashMap::new(), 2, 1) + .expect("concurrent in-place commit"); + })); + }); + let produced = dir.path().join("produced.xml"); + fs::write(&produced, VALID).expect("produced CDFI"); + + commit_designer_state(&context, &prepared(1), &produced) + .expect_err("concurrent generation must win"); + + assert_eq!( + HashStorage::new(context.storage_path()) + .current_generation() + .expect("concurrent generation retained"), + 2 + ); + } + + #[test] + fn recoverable_publication_preserves_database_published_after_claim() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + let storage_path = context.storage_path(); + fs::create_dir_all(storage_path.parent().expect("state dir")).expect("state dir"); + fs::write(&storage_path, b"corrupt-a").expect("corrupt A"); + fs::write(context.private_cdfi_path(), VALID).expect("private CDFI"); + let observed = HashStorage::new(storage_path.clone()) + .recoverable_observation() + .expect("recoverable token"); + let healthy = dir.path().join("healthy-after-claim.redb"); + HashStorage::create_replacement(healthy.clone(), &HashMap::new(), 7, 9).expect("healthy B"); + let healthy_bytes = fs::read(&healthy).expect("healthy bytes"); + let target = storage_path.clone(); + AFTER_REDB_CLAIM_HOOK.with(|cell| { + *cell.borrow_mut() = Some(Box::new(move || { + fs::hard_link(&healthy, &target).expect("publish B after claim"); + })); + }); + let produced = dir.path().join("produced.xml"); + fs::write(&produced, VALID).expect("produced CDFI"); + let mut update = prepared(0); + update.observed_storage = observed; + + commit_designer_state(&context, &update, &produced) + .expect_err("concurrent B must reject publication"); + + assert_eq!(fs::read(storage_path).expect("preserved B"), healthy_bytes); + } + + #[test] + fn recoverable_rollback_preserves_database_published_before_restore() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + let storage_path = context.storage_path(); + fs::create_dir_all(storage_path.parent().expect("state dir")).expect("state dir"); + fs::write(&storage_path, b"corrupt-a").expect("corrupt A"); + fs::write(context.private_cdfi_path(), VALID).expect("private CDFI"); + let observed = HashStorage::new(storage_path.clone()) + .recoverable_observation() + .expect("recoverable token"); + let healthy = dir.path().join("healthy-before-rollback.redb"); + HashStorage::create_replacement(healthy.clone(), &HashMap::new(), 7, 9).expect("healthy B"); + let healthy_bytes = fs::read(&healthy).expect("healthy bytes"); + let target = storage_path.clone(); + FORCE_REDB_PUBLISH_FAILURE.with(|forced| forced.set(true)); + BEFORE_REDB_ROLLBACK_HOOK.with(|cell| { + *cell.borrow_mut() = Some(Box::new(move || { + fs::hard_link(&healthy, &target).expect("publish B before rollback"); + })); + }); + let produced = dir.path().join("produced.xml"); + fs::write(&produced, VALID).expect("produced CDFI"); + let mut update = prepared(0); + update.observed_storage = observed; + + commit_designer_state(&context, &update, &produced) + .expect_err("forced publication failure"); + + assert_eq!(fs::read(storage_path).expect("preserved B"), healthy_bytes); + } + + #[test] + fn recovery_waits_while_another_owner_holds_source_state_lock() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + let lock = lock_designer_state(&context).expect("first lock"); + let (sender, receiver) = mpsc::channel(); + let worker_context = context.clone(); + let worker = std::thread::spawn(move || { + let result = recover_designer_state(&worker_context); + sender.send(result).expect("send recovery result"); + }); + + assert!(receiver.recv_timeout(Duration::from_millis(150)).is_err()); + drop(lock); + receiver + .recv_timeout(Duration::from_secs(2)) + .expect("recovery unblocked") + .expect("recovery succeeds"); + worker.join().expect("worker"); + } + + #[test] + fn stale_generation_and_invalid_output_publish_nothing() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + seed_state(&context, VALID); + let old_redb = fs::read(context.storage_path()).expect("old redb"); + let old_cdfi = fs::read(context.private_cdfi_path()).expect("old CDFI"); + let valid = dir.path().join("valid.xml"); + fs::write(&valid, VALID).expect("valid output"); + + commit_designer_state(&context, &prepared(0), &valid).expect_err("stale generation"); + assert_eq!(fs::read(context.storage_path()).expect("redb"), old_redb); + assert_eq!( + fs::read(context.private_cdfi_path()).expect("CDFI"), + old_cdfi + ); + + let invalid = dir.path().join("invalid.xml"); + fs::write(&invalid, "").expect("invalid output"); + commit_designer_state(&context, &prepared(1), &invalid).expect_err("invalid produced CDFI"); + assert_eq!(fs::read(context.storage_path()).expect("redb"), old_redb); + assert_eq!( + fs::read(context.private_cdfi_path()).expect("CDFI"), + old_cdfi + ); + } + + #[test] + fn restart_recovery_rolls_back_incomplete_pair_publication() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + seed_state(&context, VALID); + let old_redb = fs::read(context.storage_path()).expect("old redb"); + let old_cdfi = fs::read(context.private_cdfi_path()).expect("old CDFI"); + let transaction = context.transactions_dir().join("state-interrupted"); + fs::create_dir_all(&transaction).expect("transaction"); + fs::write(transaction.join(BACKUP_REDB), &old_redb).expect("redb backup"); + fs::write(transaction.join(BACKUP_CDFI), &old_cdfi).expect("CDFI backup"); + fs::write(transaction.join(STAGED_REDB), b"new partial redb").expect("staged redb"); + fs::write(transaction.join(STAGED_CDFI), b"new partial CDFI").expect("staged CDFI"); + fs::write( + transaction.join(format!("{STAGED_REDB}.original.claimed")), + &old_redb, + ) + .expect("claimed old redb"); + fs::write( + transaction.join(format!("{STAGED_CDFI}.original.claimed")), + &old_cdfi, + ) + .expect("claimed old CDFI"); + fs::remove_file(context.storage_path()).expect("remove old redb"); + fs::hard_link(transaction.join(STAGED_REDB), context.storage_path()) + .expect("publish partial redb"); + fs::remove_file(context.private_cdfi_path()).expect("remove old CDFI"); + fs::hard_link(transaction.join(STAGED_CDFI), context.private_cdfi_path()) + .expect("publish partial CDFI"); + fs::write( + transaction.join(JOURNAL_FILE), + serde_json::to_vec(&prepared_journal(&transaction, true, true)).expect("journal"), + ) + .expect("journal file"); + + recover_designer_state(&context).expect("recover"); + + assert_eq!(fs::read(context.storage_path()).expect("redb"), old_redb); + assert_eq!( + fs::read(context.private_cdfi_path()).expect("CDFI"), + old_cdfi + ); + assert!(!transaction.exists()); + } + + #[cfg(unix)] + #[test] + fn recovery_preserves_journal_when_storage_lookup_hard_fails() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + let transaction = context.transactions_dir().join("state-hard-lookup"); + fs::create_dir_all(&transaction).expect("transaction"); + fs::write(transaction.join(STAGED_REDB), b"staged-redb").expect("staged redb"); + fs::write(transaction.join(STAGED_CDFI), b"staged-cdfi").expect("staged cdfi"); + fs::write( + transaction.join(JOURNAL_FILE), + serde_json::to_vec(&prepared_journal(&transaction, false, false)).expect("journal"), + ) + .expect("journal file"); + std::os::unix::fs::symlink(context.storage_path(), context.storage_path()) + .expect("self symlink"); + + let error = recover_designer_state(&context).expect_err("hard lookup must fail closed"); + + assert!(matches!( + error, + RuntimeStateError::Storage( + crate::change_detection::hash_storage::StorageError::Hard { .. } + ) + )); + assert!(transaction.exists()); + assert!(transaction.join(JOURNAL_FILE).exists()); + assert!(transaction.join(STAGED_REDB).exists()); + assert!(transaction.join(STAGED_CDFI).exists()); + } + + #[test] + fn redb_backup_does_not_alias_the_claimed_live_database() { + let dir = tempdir().expect("tempdir"); + let target = dir.path().join("hash-storage.redb"); + let backup = dir.path().join(BACKUP_REDB); + let staged = dir.path().join(STAGED_REDB); + HashStorage::create_replacement(target.clone(), &HashMap::new(), 1, 1) + .expect("seed storage"); + fs::write(&staged, b"new staged redb").expect("staged redb"); + let original_bytes = fs::read(&target).expect("original bytes"); + + prepare_redb_backup( + &target, + &backup, + &staged, + &ObservedStorageState::Initialized { generation: 1 }, + ) + .expect("prepare independent backup"); + let original_claim = transaction_original_claim_path(&staged).expect("claim path"); + fs::write(&original_claim, b"mutated claimed database").expect("mutate claim"); + + assert_eq!(fs::read(backup).expect("backup bytes"), original_bytes); + } + + #[test] + fn restart_recovery_restores_concurrent_file_from_persisted_rollback_claim() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + seed_state(&context, VALID); + let old_redb = fs::read(context.storage_path()).expect("old redb"); + let transaction = context.transactions_dir().join("state-interrupted-claim"); + fs::create_dir_all(&transaction).expect("transaction"); + fs::write(transaction.join(BACKUP_REDB), old_redb).expect("redb backup"); + fs::write(transaction.join(STAGED_REDB), b"our staged redb").expect("staged redb"); + fs::write(transaction.join(STAGED_CDFI), b"our staged CDFI").expect("staged CDFI"); + let concurrent = b"concurrent healthy B"; + fs::write( + transaction.join(format!("{STAGED_REDB}.original.claimed")), + b"old claimed A", + ) + .expect("persisted original claim"); + fs::write(context.storage_path(), concurrent).expect("publish concurrent B"); + fs::write( + transaction.join(JOURNAL_FILE), + serde_json::to_vec(&prepared_journal(&transaction, true, false)).expect("journal"), + ) + .expect("journal file"); + + recover_designer_state(&context).expect("recover concurrent claim"); + + assert_eq!( + fs::read(context.storage_path()).expect("restored concurrent B"), + concurrent + ); + } + + #[test] + fn restart_recovery_restores_foreign_file_claimed_before_validation() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + seed_state(&context, VALID); + let old_redb = fs::read(context.storage_path()).expect("old redb"); + let old_cdfi = fs::read(context.private_cdfi_path()).expect("old CDFI"); + let transaction = context.transactions_dir().join("state-foreign-claim"); + fs::create_dir_all(&transaction).expect("transaction"); + fs::write(transaction.join(BACKUP_REDB), old_redb).expect("redb backup"); + fs::write(transaction.join(BACKUP_CDFI), old_cdfi).expect("CDFI backup"); + fs::write(transaction.join(STAGED_REDB), b"our staged redb").expect("staged redb"); + fs::write(transaction.join(STAGED_CDFI), b"our staged CDFI").expect("staged CDFI"); + let concurrent = b"concurrent CDFI B"; + fs::write( + transaction.join(format!("{STAGED_CDFI}.original.claimed")), + concurrent, + ) + .expect("persisted foreign claim"); + fs::remove_file(context.private_cdfi_path()).expect("crash after CDFI claim"); + fs::write( + transaction.join(JOURNAL_FILE), + serde_json::to_vec(&prepared_journal(&transaction, true, true)).expect("journal"), + ) + .expect("journal file"); + + recover_designer_state(&context).expect("recover foreign claim"); + + assert_eq!( + fs::read(context.private_cdfi_path()).expect("restored foreign CDFI"), + concurrent + ); + } + + #[test] + fn restart_recovery_preserves_same_bytes_foreign_original_claims() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + seed_state(&context, VALID); + let old_redb = fs::read(context.storage_path()).expect("old redb"); + let old_cdfi = fs::read(context.private_cdfi_path()).expect("old CDFI"); + let transaction = context + .transactions_dir() + .join("state-same-bytes-originals"); + fs::create_dir_all(&transaction).expect("transaction"); + fs::write(transaction.join(BACKUP_REDB), &old_redb).expect("redb backup"); + fs::write(transaction.join(BACKUP_CDFI), &old_cdfi).expect("CDFI backup"); + fs::write(transaction.join(STAGED_REDB), b"new redb").expect("staged redb"); + fs::write(transaction.join(STAGED_CDFI), b"new CDFI").expect("staged CDFI"); + let redb_claim = transaction.join(format!("{STAGED_REDB}.original.claimed")); + let cdfi_claim = transaction.join(format!("{STAGED_CDFI}.original.claimed")); + fs::copy(transaction.join(BACKUP_REDB), &redb_claim) + .expect("same-bytes foreign redb claim"); + fs::copy(transaction.join(BACKUP_CDFI), &cdfi_claim) + .expect("same-bytes foreign CDFI claim"); + let redb_identity = file_fingerprint(&redb_claim) + .expect("redb claim fingerprint") + .identity; + let cdfi_identity = file_fingerprint(&cdfi_claim) + .expect("CDFI claim fingerprint") + .identity; + fs::remove_file(context.storage_path()).expect("crash after redb claim"); + fs::remove_file(context.private_cdfi_path()).expect("crash after CDFI claim"); + fs::write( + transaction.join(JOURNAL_FILE), + serde_json::to_vec(&prepared_journal(&transaction, true, true)).expect("journal"), + ) + .expect("journal file"); + + recover_designer_state(&context).expect("recover same-bytes original claims"); + + assert_eq!( + file_fingerprint(&context.storage_path()) + .expect("restored redb fingerprint") + .identity, + redb_identity + ); + assert_eq!( + file_fingerprint(&context.private_cdfi_path()) + .expect("restored CDFI fingerprint") + .identity, + cdfi_identity + ); + } + + #[test] + fn restart_recovery_preserves_same_bytes_foreign_redb_and_cdfi() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + seed_state(&context, VALID); + let old_redb = fs::read(context.storage_path()).expect("old redb"); + let old_cdfi = fs::read(context.private_cdfi_path()).expect("old CDFI"); + let transaction = context.transactions_dir().join("state-same-bytes-foreign"); + fs::create_dir_all(&transaction).expect("transaction"); + fs::write(transaction.join(BACKUP_REDB), &old_redb).expect("redb backup"); + fs::write(transaction.join(BACKUP_CDFI), &old_cdfi).expect("CDFI backup"); + let new_redb = b"same bytes redb"; + let new_cdfi = b"same bytes CDFI"; + fs::write(transaction.join(STAGED_REDB), new_redb).expect("staged redb"); + fs::write(transaction.join(STAGED_CDFI), new_cdfi).expect("staged CDFI"); + fs::write( + transaction.join(format!("{STAGED_REDB}.original.claimed")), + &old_redb, + ) + .expect("redb original claim"); + fs::write( + transaction.join(format!("{STAGED_CDFI}.original.claimed")), + &old_cdfi, + ) + .expect("CDFI original claim"); + fs::remove_file(context.storage_path()).expect("remove old redb"); + fs::copy(transaction.join(STAGED_REDB), context.storage_path()) + .expect("independent same-bytes redb B"); + fs::remove_file(context.private_cdfi_path()).expect("remove old CDFI"); + fs::copy(transaction.join(STAGED_CDFI), context.private_cdfi_path()) + .expect("independent same-bytes CDFI B"); + fs::write( + transaction.join(JOURNAL_FILE), + serde_json::to_vec(&prepared_journal(&transaction, true, true)).expect("journal"), + ) + .expect("journal file"); + + recover_designer_state(&context).expect("recover same-bytes foreign files"); + + assert_eq!( + fs::read(context.storage_path()).expect("foreign redb"), + new_redb + ); + assert_eq!( + fs::read(context.private_cdfi_path()).expect("foreign CDFI"), + new_cdfi + ); + } + + #[test] + fn restart_recovery_rolls_back_staged_target_when_original_claim_persisted() { + let dir = tempdir().expect("tempdir"); + let context = context(dir.path()); + seed_state(&context, VALID); + let old_redb = fs::read(context.storage_path()).expect("old redb"); + let old_cdfi = fs::read(context.private_cdfi_path()).expect("old CDFI"); + let transaction = context.transactions_dir().join("state-after-redb-publish"); + fs::create_dir_all(&transaction).expect("transaction"); + fs::write(transaction.join(BACKUP_REDB), &old_redb).expect("redb backup"); + fs::write(transaction.join(BACKUP_CDFI), &old_cdfi).expect("CDFI backup"); + fs::write(transaction.join(STAGED_REDB), b"new staged redb").expect("staged redb"); + fs::write(transaction.join(STAGED_CDFI), b"new staged CDFI").expect("staged CDFI"); + fs::write( + transaction.join(format!("{STAGED_REDB}.original.claimed")), + &old_redb, + ) + .expect("persisted original claim"); + fs::remove_file(context.storage_path()).expect("remove old redb"); + fs::hard_link(transaction.join(STAGED_REDB), context.storage_path()) + .expect("publish staged redb"); + fs::write( + transaction.join(JOURNAL_FILE), + serde_json::to_vec(&prepared_journal(&transaction, true, true)).expect("journal"), + ) + .expect("journal file"); + + recover_designer_state(&context).expect("recover after staged publication"); + + assert_eq!( + fs::read(context.storage_path()).expect("old redb"), + old_redb + ); + assert_eq!( + fs::read_to_string(context.private_cdfi_path()).expect("old CDFI"), + VALID + ); + } +} diff --git a/src/use_cases/shadow_merge.rs b/src/use_cases/shadow_merge.rs new file mode 100644 index 0000000..98aa82e --- /dev/null +++ b/src/use_cases/shadow_merge.rs @@ -0,0 +1,531 @@ +use std::collections::{BTreeMap, BTreeSet}; + +use thiserror::Error; + +use crate::domain::sync_receipt::{SyncReceipt, SyncReceiptError, SyncTarget}; +use crate::use_cases::dump_execution::ShadowWriteSet; + +/// Raw SHA-256 hash of one managed file version. +pub(crate) type RawFileHash = [u8; 32]; + +/// One file version observed at a point in the three-way merge. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum FileVersion { + Absent, + Present(RawFileHash), +} + +/// Publication decision for one managed path. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum MergeAction { + /// Source and baseline match, so the dump version must be published. + Apply, + /// Source and dump independently reached the same version. + Converged, + /// The dump is unchanged, so the local source version must be preserved. + RetainLocal, + /// Source and dump diverged from the baseline and from each other. + Conflict, + /// All three versions are identical. + NoOp, +} + +/// One deterministic path-level decision in a complete manifest merge. +#[derive(Debug, Clone, PartialEq, Eq)] +pub(crate) struct ManifestMergeEntry { + path: String, + baseline: FileVersion, + source: FileVersion, + dump: FileVersion, + action: MergeAction, +} + +impl ManifestMergeEntry { + pub(crate) fn path(&self) -> &str { + &self.path + } + + pub(crate) const fn baseline(&self) -> FileVersion { + self.baseline + } + + pub(crate) const fn source(&self) -> FileVersion { + self.source + } + + pub(crate) const fn dump(&self) -> FileVersion { + self.dump + } + + pub(crate) const fn action(&self) -> MergeAction { + self.action + } +} + +/// Sorted complete merge plan. Callers must reject the whole plan when it has conflicts. +#[derive(Debug, Clone, PartialEq, Eq)] +pub(crate) struct ManifestMergePlan { + entries: Vec, +} + +impl ManifestMergePlan { + pub(crate) fn entries(&self) -> &[ManifestMergeEntry] { + &self.entries + } + + pub(crate) fn has_conflicts(&self) -> bool { + self.entries + .iter() + .any(|entry| entry.action == MergeAction::Conflict) + } + + pub(crate) fn applied_receipt( + &self, + writes: &ShadowWriteSet, + ) -> Result { + if self.has_conflicts() { + return Err(MergeReceiptError::ConflictedPlan); + } + let requested = self.requested_targets()?; + let processed = writes + .paths() + .iter() + .map(|path| { + let entry = self + .entries + .binary_search_by(|entry| entry.path.as_str().cmp(path)) + .ok() + .map(|index| &self.entries[index]) + .ok_or_else(|| MergeReceiptError::UnknownProcessedPath(path.clone()))?; + sync_target(entry)? + .ok_or_else(|| MergeReceiptError::UnknownProcessedPath(path.clone())) + }) + .collect::, _>>()?; + let mut skipped = Vec::new(); + for entry in &self.entries { + let Some(target) = sync_target(entry)? else { + continue; + }; + match entry.action { + MergeAction::Apply | MergeAction::Converged => {} + MergeAction::RetainLocal | MergeAction::NoOp => skipped.push(target), + MergeAction::Conflict => return Err(MergeReceiptError::ConflictedPlan), + } + } + Ok(SyncReceipt::applied(requested, processed, skipped)?) + } + + pub(crate) fn failed_receipt(&self) -> Result { + Ok(SyncReceipt::failed(self.requested_targets()?)?) + } + + pub(crate) fn conflict_receipt(&self) -> Result { + if !self.has_conflicts() { + return Err(MergeReceiptError::ConflictFreePlan); + } + let requested = self.requested_targets()?; + let mut skipped = Vec::new(); + let mut conflicted = Vec::new(); + for entry in &self.entries { + let Some(target) = sync_target(entry)? else { + continue; + }; + match entry.action { + MergeAction::Conflict => conflicted.push(target), + MergeAction::Apply + | MergeAction::Converged + | MergeAction::RetainLocal + | MergeAction::NoOp => skipped.push(target), + } + } + Ok(SyncReceipt::conflict(requested, skipped, conflicted)?) + } + + fn requested_targets(&self) -> Result, MergeReceiptError> { + let mut targets = Vec::new(); + for entry in self + .entries + .iter() + .filter(|entry| entry.source != entry.dump) + { + if let Some(target) = sync_target(entry)? { + targets.push(target); + } + } + Ok(targets) + } +} + +#[derive(Debug, Error)] +pub(crate) enum MergeReceiptError { + #[error("cannot create an applied receipt for a conflicted merge plan")] + ConflictedPlan, + #[error("cannot create a conflict receipt for a conflict-free merge plan")] + ConflictFreePlan, + #[error("platform write evidence contains unknown managed path '{0}'")] + UnknownProcessedPath(String), + #[error("invalid merge receipt: {0}")] + InvalidReceipt(#[from] SyncReceiptError), +} + +fn sync_target(entry: &ManifestMergeEntry) -> Result, SyncReceiptError> { + let pre_hash = version_hash(entry.source); + let post_hash = version_hash(entry.dump); + if pre_hash.is_none() && post_hash.is_none() { + return Ok(None); + } + SyncTarget::new(&entry.path, pre_hash, post_hash).map(Some) +} + +fn version_hash(version: FileVersion) -> Option { + match version { + FileVersion::Absent => None, + FileVersion::Present(hash) => Some( + hash.iter() + .map(|byte| format!("{byte:02x}")) + .collect::(), + ), + } +} + +/// Plan all paths in the union of the three complete manifests. +#[must_use] +pub(crate) fn plan_manifest_merge( + baseline: &BTreeMap, + source: &BTreeMap, + dump: &BTreeMap, +) -> ManifestMergePlan { + let paths = baseline + .keys() + .chain(source.keys()) + .chain(dump.keys()) + .cloned() + .collect::>(); + let entries = paths + .into_iter() + .map(|path| { + let baseline = file_version(baseline, &path); + let source = file_version(source, &path); + let dump = file_version(dump, &path); + ManifestMergeEntry { + action: plan_file_merge(baseline, source, dump), + path, + baseline, + source, + dump, + } + }) + .collect(); + ManifestMergePlan { entries } +} + +fn file_version(manifest: &BTreeMap, path: &str) -> FileVersion { + match manifest.get(path) { + Some(hash) => FileVersion::Present(*hash), + None => FileVersion::Absent, + } +} + +/// Plan publication for one managed path from baseline (`B`), source (`S`) and dump (`D`). +#[must_use] +pub(crate) fn plan_file_merge( + baseline: FileVersion, + source: FileVersion, + dump: FileVersion, +) -> MergeAction { + if baseline == FileVersion::Absent + && source != FileVersion::Absent + && dump == FileVersion::Absent + { + MergeAction::Conflict + } else if source == baseline && dump == baseline { + MergeAction::NoOp + } else if source == baseline { + MergeAction::Apply + } else if dump == baseline { + MergeAction::RetainLocal + } else if source == dump { + MergeAction::Converged + } else { + MergeAction::Conflict + } +} + +#[cfg(test)] +mod tests { + use std::collections::BTreeMap; + + use super::{plan_file_merge, plan_manifest_merge, FileVersion, MergeAction, RawFileHash}; + use crate::use_cases::dump_execution::ShadowWriteSet; + + const BASELINE: [u8; 32] = [0x11; 32]; + const SOURCE: [u8; 32] = [0x22; 32]; + const DUMP: [u8; 32] = [0x33; 32]; + + #[test] + fn plans_every_present_baseline_equality_combination() { + let cases = [ + ( + "all versions equal", + present(BASELINE), + present(BASELINE), + present(BASELINE), + MergeAction::NoOp, + ), + ( + "dump modifies unchanged source", + present(BASELINE), + present(BASELINE), + present(DUMP), + MergeAction::Apply, + ), + ( + "dump deletes unchanged source", + present(BASELINE), + present(BASELINE), + FileVersion::Absent, + MergeAction::Apply, + ), + ( + "source modifies unchanged dump", + present(BASELINE), + present(SOURCE), + present(BASELINE), + MergeAction::RetainLocal, + ), + ( + "source deletes unchanged dump", + present(BASELINE), + FileVersion::Absent, + present(BASELINE), + MergeAction::RetainLocal, + ), + ( + "source and dump converge on content", + present(BASELINE), + present(SOURCE), + present(SOURCE), + MergeAction::Converged, + ), + ( + "source and dump converge on deletion", + present(BASELINE), + FileVersion::Absent, + FileVersion::Absent, + MergeAction::Converged, + ), + ( + "source and dump modify differently", + present(BASELINE), + present(SOURCE), + present(DUMP), + MergeAction::Conflict, + ), + ( + "local modification conflicts with dump deletion", + present(BASELINE), + present(SOURCE), + FileVersion::Absent, + MergeAction::Conflict, + ), + ( + "local deletion conflicts with dump modification", + present(BASELINE), + FileVersion::Absent, + present(DUMP), + MergeAction::Conflict, + ), + ]; + + assert_cases(&cases); + } + + #[test] + fn plans_absent_baseline_bootstrap_without_overwriting_local_files() { + let cases = [ + ( + "file remains absent", + FileVersion::Absent, + FileVersion::Absent, + FileVersion::Absent, + MergeAction::NoOp, + ), + ( + "dump adds a new file", + FileVersion::Absent, + FileVersion::Absent, + present(DUMP), + MergeAction::Apply, + ), + ( + "local-only file conflicts during bootstrap", + FileVersion::Absent, + present(SOURCE), + FileVersion::Absent, + MergeAction::Conflict, + ), + ( + "matching local and dump additions converge", + FileVersion::Absent, + present(SOURCE), + present(SOURCE), + MergeAction::Converged, + ), + ( + "different local and dump additions conflict", + FileVersion::Absent, + present(SOURCE), + present(DUMP), + MergeAction::Conflict, + ), + ]; + + assert_cases(&cases); + } + + #[test] + fn plans_sorted_union_of_baseline_source_and_dump_manifests() { + let baseline = manifest(&[ + ("apply.txt", BASELINE), + ("delete.txt", BASELINE), + ("gone.txt", BASELINE), + ("local.txt", BASELINE), + ("conflict.txt", BASELINE), + ]); + let source = manifest(&[ + ("apply.txt", BASELINE), + ("delete.txt", BASELINE), + ("local.txt", SOURCE), + ("conflict.txt", SOURCE), + ("local-only.txt", SOURCE), + ]); + let dump = manifest(&[ + ("apply.txt", DUMP), + ("local.txt", BASELINE), + ("conflict.txt", DUMP), + ("created.txt", DUMP), + ]); + + let plan = plan_manifest_merge(&baseline, &source, &dump); + + assert_eq!( + plan.entries() + .iter() + .map(|entry| (entry.path(), entry.action())) + .collect::>(), + vec![ + ("apply.txt", MergeAction::Apply), + ("conflict.txt", MergeAction::Conflict), + ("created.txt", MergeAction::Apply), + ("delete.txt", MergeAction::Apply), + ("gone.txt", MergeAction::Converged), + ("local-only.txt", MergeAction::Conflict), + ("local.txt", MergeAction::RetainLocal), + ] + ); + assert!(plan.has_conflicts()); + } + + #[test] + fn builds_exact_applied_failed_and_conflict_receipts() { + let clean = plan_manifest_merge( + &manifest(&[ + ("apply.txt", BASELINE), + ("converged.txt", BASELINE), + ("local.txt", BASELINE), + ("noop.txt", BASELINE), + ]), + &manifest(&[ + ("apply.txt", BASELINE), + ("converged.txt", SOURCE), + ("local.txt", SOURCE), + ("noop.txt", BASELINE), + ]), + &manifest(&[ + ("apply.txt", DUMP), + ("converged.txt", SOURCE), + ("local.txt", BASELINE), + ("noop.txt", BASELINE), + ]), + ); + + let writes = ShadowWriteSet::from_paths_for_test(&[ + "apply.txt", + "converged.txt", + "local.txt", + "noop.txt", + ]); + let applied = + serde_json::to_value(clean.applied_receipt(&writes).expect("applied receipt")) + .expect("serialize applied"); + assert_eq!(applied["status"], "applied"); + assert_eq!( + receipt_paths(&applied, "requested"), + vec!["apply.txt", "local.txt"] + ); + assert_eq!( + receipt_paths(&applied, "processed"), + vec!["apply.txt", "converged.txt", "local.txt", "noop.txt"] + ); + assert_eq!( + receipt_paths(&applied, "skipped"), + vec!["local.txt", "noop.txt"] + ); + + let failed = serde_json::to_value(clean.failed_receipt().expect("failed receipt")) + .expect("serialize failed"); + assert_eq!(failed["status"], "failed"); + assert_eq!( + receipt_paths(&failed, "requested"), + vec!["apply.txt", "local.txt"] + ); + assert_eq!(receipt_paths(&failed, "processed"), Vec::::new()); + + let conflicted = plan_manifest_merge( + &manifest(&[("conflict.txt", BASELINE), ("safe.txt", BASELINE)]), + &manifest(&[("conflict.txt", SOURCE), ("safe.txt", BASELINE)]), + &manifest(&[("conflict.txt", DUMP), ("safe.txt", DUMP)]), + ); + let conflict = + serde_json::to_value(conflicted.conflict_receipt().expect("conflict receipt")) + .expect("serialize conflict"); + assert_eq!(conflict["status"], "conflict"); + assert_eq!( + receipt_paths(&conflict, "requested"), + vec!["conflict.txt", "safe.txt"] + ); + assert_eq!(receipt_paths(&conflict, "processed"), Vec::::new()); + assert_eq!(receipt_paths(&conflict, "skipped"), vec!["safe.txt"]); + assert_eq!(receipt_paths(&conflict, "conflicted"), vec!["conflict.txt"]); + } + + fn present(hash: [u8; 32]) -> FileVersion { + FileVersion::Present(hash) + } + + fn assert_cases(cases: &[(&str, FileVersion, FileVersion, FileVersion, MergeAction)]) { + for (name, baseline, source, dump, expected) in cases { + assert_eq!( + plan_file_merge(*baseline, *source, *dump), + *expected, + "case: {name}" + ); + } + } + + fn manifest(entries: &[(&str, RawFileHash)]) -> BTreeMap { + entries + .iter() + .map(|(path, hash)| ((*path).to_owned(), *hash)) + .collect() + } + + fn receipt_paths(value: &serde_json::Value, list: &str) -> Vec { + value[list] + .as_array() + .expect("receipt list") + .iter() + .map(|target| target["path"].as_str().expect("path").to_owned()) + .collect() + } +} diff --git a/src/use_cases/source_inventory.rs b/src/use_cases/source_inventory.rs index aa76335..9894250 100644 --- a/src/use_cases/source_inventory.rs +++ b/src/use_cases/source_inventory.rs @@ -4,6 +4,7 @@ use std::path::PathBuf; use crate::change_detection::analyzer::ContextAnalysis; use crate::change_detection::source_sets::SourceSetsService; use crate::config::model::{AppConfig, SourceSetConfig, SourceSetPurpose}; +use crate::domain::runtime_state::RuntimeStateError; use crate::domain::source_set::SourceSetContext; /// Read-only runtime index for source-set orchestration. @@ -17,12 +18,12 @@ pub(crate) struct SourceSetInventory<'a> { } impl<'a> SourceSetInventory<'a> { - pub(crate) fn new(config: &'a AppConfig) -> Self { - let service = SourceSetsService::new(config); - let designer_contexts = service.designer_contexts(); - let edt_contexts = service.edt_contexts(); + pub(crate) fn new(config: &'a AppConfig) -> Result { + let service = SourceSetsService::new(config)?; + let designer_contexts = service.designer_contexts()?; + let edt_contexts = service.edt_contexts()?; - Self { + Ok(Self { config, source_sets_by_name: config .source_sets @@ -33,7 +34,7 @@ impl<'a> SourceSetInventory<'a> { designer_contexts, edt_contexts_by_name: index_contexts(&edt_contexts), edt_contexts, - } + }) } pub(crate) fn source_sets(&self) -> Vec<&'a SourceSetConfig> { @@ -105,7 +106,7 @@ impl<'a> SourceSetInventory<'a> { } pub(crate) fn analyze_contexts(&self, contexts: &[SourceSetContext]) -> Vec { - SourceSetsService::new(self.config).analyze_contexts(contexts) + crate::change_detection::analyzer::analyze_contexts(contexts) } } @@ -168,7 +169,7 @@ mod tests { #[test] fn ordered_source_sets_group_configuration_extensions_and_external_sets() { let config = config(SourceFormat::Designer); - let inventory = SourceSetInventory::new(&config); + let inventory = SourceSetInventory::new(&config).expect("inventory"); let names = inventory .ordered_source_sets() @@ -182,7 +183,7 @@ mod tests { #[test] fn indexes_designer_and_edt_contexts_by_source_set_identity() { let config = config(SourceFormat::Edt); - let inventory = SourceSetInventory::new(&config); + let inventory = SourceSetInventory::new(&config).expect("inventory"); let main = inventory.source_set("main").expect("main source-set"); assert_eq!( @@ -198,4 +199,18 @@ mod tests { config.base_path.join("configuration").as_path() ); } + + #[test] + fn malformed_nonempty_designer_connection_returns_error_without_unwinding() { + let mut config = config(SourceFormat::Designer); + config.infobase.connection = "/Unsupported secret-value".to_owned(); + + let result = std::panic::catch_unwind(|| SourceSetInventory::new(&config)); + + assert!(result.is_ok(), "inventory construction must not unwind"); + assert!(matches!( + result.expect("no unwind"), + Err(crate::domain::runtime_state::RuntimeStateError::UnsupportedRawConnection) + )); + } } diff --git a/src/use_cases/source_publication.rs b/src/use_cases/source_publication.rs new file mode 100644 index 0000000..154e814 --- /dev/null +++ b/src/use_cases/source_publication.rs @@ -0,0 +1,2474 @@ +use std::collections::BTreeMap; +use std::fs::{self, File, OpenOptions}; +use std::io::{self, Read, Write}; +use std::path::{Component, Path, PathBuf}; + +use serde::{Deserialize, Serialize}; +use sha2::{Digest, Sha256}; +use thiserror::Error; + +use crate::change_detection::scanner::is_always_ignored_relative_path; +use crate::domain::runtime_state::DumpTransactionId; +use crate::use_cases::shadow_merge::{FileVersion, ManifestMergePlan, MergeAction, RawFileHash}; + +const JOURNAL_FILE: &str = "journal.json"; + +#[cfg(test)] +thread_local! { + static BEFORE_FIRST_MUTATION: std::cell::RefCell>> = + const { std::cell::RefCell::new(None) }; + static BEFORE_ACTION_INSTALL: std::cell::RefCell>> = + const { std::cell::RefCell::new(None) }; + static AFTER_SAFE_PARENT_OPEN: std::cell::RefCell>> = + const { std::cell::RefCell::new(None) }; + static FAIL_JOURNAL_STATUS: std::cell::Cell> = + const { std::cell::Cell::new(None) }; +} + +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(transparent)] +pub(crate) struct TargetIdentity(String); + +impl TargetIdentity { + pub(crate) fn new(value: impl Into) -> Self { + Self(value.into()) + } +} + +#[derive(Debug, Clone, PartialEq, Eq)] +pub(crate) struct ObservedStateGeneration { + generation: u64, + dump_transaction_id: Option, +} + +impl ObservedStateGeneration { + pub(crate) const fn new(value: u64) -> Self { + Self { + generation: value, + dump_transaction_id: None, + } + } + + pub(crate) fn with_dump_transaction( + generation: u64, + dump_transaction_id: DumpTransactionId, + ) -> Self { + Self { + generation, + dump_transaction_id: Some(dump_transaction_id), + } + } +} + +#[derive(Debug, Error)] +pub(crate) enum PublicationError { + #[cfg(not(any(unix, windows)))] + #[error("safe descriptor-relative source publication is unavailable on this platform")] + UnsupportedSafeFilesystem, + #[error("manifest merge contains a conflict")] + Conflict, + #[error("publication request is missing required field '{field}'")] + MissingField { field: &'static str }, + #[error("invalid managed publication path '{path}'")] + InvalidPath { path: String }, + #[error("managed snapshot changed at '{path}' before publication")] + SnapshotMismatch { path: String }, + #[error("managed path '{path}' is not a regular file")] + UnsafeFile { path: PathBuf }, + #[error("publication journal target does not match this source or target identity")] + ForeignTarget, + #[error("foreign content at '{path}' prevents safe publication recovery")] + ForeignContent { path: PathBuf }, + #[error("invalid publication journal '{path}': {reason}")] + InvalidJournal { path: PathBuf, reason: String }, + #[error("failed to {operation} '{path}': {source}")] + Io { + operation: &'static str, + path: PathBuf, + source: io::Error, + }, + #[error("source publication failed ({publication}); rollback also failed ({rollback})")] + PublicationAndRollback { + publication: Box, + rollback: Box, + }, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +enum JournalStatus { + Prepared, + SourceApplied, + StateVisible, + Committed, +} + +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +struct ExpectedFile { + sha256: Option, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +struct JournalAction { + path: String, + before: ExpectedFile, + after: ExpectedFile, + backup: Option, + payload: Option, + claim: String, +} + +#[derive(Debug, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +struct PublicationJournal { + generation: u64, + target_identity: TargetIdentity, + source_root: PathBuf, + status: JournalStatus, + actions: Vec, + dump_transaction_id: DumpTransactionId, +} + +pub(crate) struct PublicationRequest<'a> { + source_root: &'a Path, + dump_root: &'a Path, + plan: &'a ManifestMergePlan, + transaction_root: &'a Path, + generation: u64, + target_identity: TargetIdentity, + dump_transaction_id: DumpTransactionId, +} + +impl<'a> PublicationRequest<'a> { + pub(crate) fn builder( + source_root: &'a Path, + dump_root: &'a Path, + plan: &'a ManifestMergePlan, + ) -> PublicationRequestBuilder<'a> { + PublicationRequestBuilder { + source_root, + dump_root, + plan, + transaction_root: None, + generation: None, + target_identity: None, + dump_transaction_id: None, + } + } + + pub(crate) fn prepare(self) -> Result { + #[cfg(not(any(unix, windows)))] + return Err(PublicationError::UnsupportedSafeFilesystem); + #[cfg(any(unix, windows))] + { + if self.plan.has_conflicts() { + return Err(PublicationError::Conflict); + } + let source_root = canonical_directory(self.source_root)?; + let dump_root = canonical_directory(self.dump_root)?; + let mut observed_source = BTreeMap::new(); + let mut observed_dump = BTreeMap::new(); + + // This is deliberately one complete validation pass before transaction creation or + // source mutation. Opened files are no-follow and their bytes provide the payloads. + for entry in self.plan.entries() { + validate_relative(entry.path())?; + let source = read_expected(&source_root, entry.path(), entry.source())?; + observed_source.insert(entry.path().to_owned(), source); + if entry.action() == MergeAction::Apply { + let dump = read_expected(&dump_root, entry.path(), entry.dump())?; + observed_dump.insert(entry.path().to_owned(), dump); + } + } + + if self + .transaction_root + .try_exists() + .map_err(|source| io_error("inspect transaction", self.transaction_root, source))? + { + return Err(PublicationError::InvalidJournal { + path: self.transaction_root.to_path_buf(), + reason: "transaction already exists and must be recovered first".to_owned(), + }); + } + fs::create_dir(self.transaction_root) + .map_err(|source| io_error("create transaction", self.transaction_root, source))?; + let payload_root = self.transaction_root.join("payloads"); + let backup_root = self.transaction_root.join("backups"); + fs::create_dir(&payload_root) + .map_err(|source| io_error("create payload directory", &payload_root, source))?; + fs::create_dir(&backup_root) + .map_err(|source| io_error("create backup directory", &backup_root, source))?; + + let mut actions = Vec::new(); + for entry in self + .plan + .entries() + .iter() + .filter(|entry| entry.action() == MergeAction::Apply) + { + let index = actions.len(); + let before_bytes = observed_source.remove(entry.path()).flatten(); + let after_bytes = observed_dump.remove(entry.path()).flatten(); + let backup = write_blob(&backup_root, index, before_bytes.as_deref())?; + let payload = write_blob(&payload_root, index, after_bytes.as_deref())?; + actions.push(JournalAction { + path: entry.path().to_owned(), + before: expected(before_bytes.as_deref()), + after: expected(after_bytes.as_deref()), + backup, + payload, + claim: format!(".v8-runner-publication-claim-{}", uuid::Uuid::new_v4()), + }); + } + sync_directory(&payload_root)?; + sync_directory(&backup_root)?; + let journal = PublicationJournal { + generation: self.generation, + target_identity: self.target_identity, + source_root, + status: JournalStatus::Prepared, + actions, + dump_transaction_id: self.dump_transaction_id, + }; + write_journal(self.transaction_root, &journal)?; + sync_directory(self.transaction_root)?; + if let Some(parent) = self.transaction_root.parent() { + sync_directory(parent)?; + } + Ok(PreparedPublication { + transaction_root: self.transaction_root.to_path_buf(), + journal, + }) + } + } +} + +pub(crate) struct PublicationRequestBuilder<'a> { + source_root: &'a Path, + dump_root: &'a Path, + plan: &'a ManifestMergePlan, + transaction_root: Option<&'a Path>, + generation: Option, + target_identity: Option, + dump_transaction_id: Option, +} + +impl<'a> PublicationRequestBuilder<'a> { + pub(crate) fn transaction_root(mut self, value: &'a Path) -> Self { + self.transaction_root = Some(value); + self + } + + pub(crate) fn generation(mut self, value: u64) -> Self { + self.generation = Some(value); + self + } + + pub(crate) fn target_identity(mut self, value: TargetIdentity) -> Self { + self.target_identity = Some(value); + self + } + + pub(crate) fn dump_transaction_id(mut self, value: DumpTransactionId) -> Self { + self.dump_transaction_id = Some(value); + self + } + + pub(crate) fn prepare(self) -> Result { + PublicationRequest { + source_root: self.source_root, + dump_root: self.dump_root, + plan: self.plan, + transaction_root: self + .transaction_root + .ok_or(PublicationError::MissingField { + field: "transaction_root", + })?, + generation: self.generation.ok_or(PublicationError::MissingField { + field: "generation", + })?, + target_identity: self.target_identity.ok_or(PublicationError::MissingField { + field: "target_identity", + })?, + dump_transaction_id: self.dump_transaction_id.ok_or( + PublicationError::MissingField { + field: "dump_transaction_id", + }, + )?, + } + .prepare() + } +} + +#[derive(Debug)] +pub(crate) struct PreparedPublication { + transaction_root: PathBuf, + journal: PublicationJournal, +} + +impl PreparedPublication { + pub(crate) fn apply(mut self) -> Result { + if let Err(error) = apply_forward(&self.transaction_root, &self.journal) { + return match rollback(&self.transaction_root, &self.journal) { + Ok(()) => Err(error), + Err(rollback) => Err(PublicationError::PublicationAndRollback { + publication: Box::new(error), + rollback: Box::new(rollback), + }), + }; + } + self.journal.status = JournalStatus::SourceApplied; + if let Err(publication) = write_journal(&self.transaction_root, &self.journal) { + return match rollback(&self.transaction_root, &self.journal) { + Ok(()) => Err(publication), + Err(rollback) => Err(PublicationError::PublicationAndRollback { + publication: Box::new(publication), + rollback: Box::new(rollback), + }), + }; + } + Ok(SourceAppliedPublication { + transaction_root: self.transaction_root, + journal: self.journal, + }) + } + + #[cfg(test)] + fn leave_for_recovery(self) {} +} + +#[derive(Debug)] +pub(crate) struct SourceAppliedPublication { + transaction_root: PathBuf, + journal: PublicationJournal, +} + +impl SourceAppliedPublication { + /// Call only after the corresponding private state generation is durably visible. + pub(crate) fn mark_state_visible( + mut self, + observed: ObservedStateGeneration, + ) -> Result { + if observed.generation != self.journal.generation + || observed.dump_transaction_id.as_ref() != Some(&self.journal.dump_transaction_id) + { + return Err(PublicationError::InvalidJournal { + path: self.transaction_root.join(JOURNAL_FILE), + reason: format!( + "state generation {} is visible, expected {}", + observed.generation, self.journal.generation + ), + }); + } + self.journal.status = JournalStatus::StateVisible; + write_journal(&self.transaction_root, &self.journal)?; + Ok(StateVisiblePublication { + transaction_root: self.transaction_root, + journal: self.journal, + }) + } + + #[cfg(test)] + fn leave_for_recovery(self) {} +} + +#[derive(Debug)] +pub(crate) struct StateVisiblePublication { + transaction_root: PathBuf, + journal: PublicationJournal, +} + +impl StateVisiblePublication { + pub(crate) fn commit(mut self) -> Result<(), PublicationError> { + self.journal.status = JournalStatus::Committed; + write_journal(&self.transaction_root, &self.journal)?; + cleanup_transaction(&self.transaction_root) + } + + #[cfg(test)] + fn leave_for_recovery(self) {} + + #[cfg(test)] + fn mark_committed_for_recovery(mut self) { + self.journal.status = JournalStatus::Committed; + write_journal(&self.transaction_root, &self.journal).unwrap(); + } +} + +pub(crate) fn recover_publication( + transaction_root: &Path, + source_root: &Path, + target_identity: &TargetIdentity, + observed: ObservedStateGeneration, +) -> Result<(), PublicationError> { + #[cfg(not(any(unix, windows)))] + return Err(PublicationError::UnsupportedSafeFilesystem); + #[cfg(any(unix, windows))] + { + if !transaction_root + .try_exists() + .map_err(|source| io_error("inspect transaction", transaction_root, source))? + { + return Ok(()); + } + let journal = read_journal(transaction_root)?; + let source_root = canonical_directory(source_root)?; + if journal.source_root != source_root || &journal.target_identity != target_identity { + return Err(PublicationError::ForeignTarget); + } + match journal.status { + JournalStatus::Prepared | JournalStatus::SourceApplied => { + if observed.generation == journal.generation + && observed.dump_transaction_id.as_ref() == Some(&journal.dump_transaction_id) + { + let mut committed = journal; + committed.status = JournalStatus::Committed; + write_journal(transaction_root, &committed)?; + } else { + rollback(transaction_root, &journal)?; + } + } + JournalStatus::StateVisible => { + if observed.generation != journal.generation + || observed.dump_transaction_id.as_ref() != Some(&journal.dump_transaction_id) + { + return Err(PublicationError::ForeignTarget); + } + let mut committed = journal; + committed.status = JournalStatus::Committed; + write_journal(transaction_root, &committed)?; + } + JournalStatus::Committed => {} + } + cleanup_transaction(transaction_root) + } +} + +fn apply_forward( + transaction_root: &Path, + journal: &PublicationJournal, +) -> Result<(), PublicationError> { + #[cfg(windows)] + reconcile_windows_scratch(journal, Direction::Forward)?; + for action in &journal.actions { + if inspect_file(&journal.source_root, &action.path)? != action.before { + return Err(PublicationError::ForeignContent { + path: journal.source_root.join(&action.path), + }); + } + } + #[cfg(test)] + BEFORE_FIRST_MUTATION.with(|hook| { + if let Some(hook) = hook.borrow_mut().take() { + hook(); + } + }); + for action in &journal.actions { + let current = inspect_file(&journal.source_root, &action.path)?; + if current != action.before { + return Err(PublicationError::ForeignContent { + path: journal.source_root.join(&action.path), + }); + } + #[cfg(test)] + BEFORE_ACTION_INSTALL.with(|hook| { + if let Some(hook) = hook.borrow_mut().take() { + hook(); + } + }); + set_file(transaction_root, journal, action, Direction::Forward)?; + } + sync_directory(&journal.source_root) +} + +fn rollback(transaction_root: &Path, journal: &PublicationJournal) -> Result<(), PublicationError> { + #[cfg(windows)] + reconcile_windows_scratch(journal, Direction::Backward)?; + let mut foreign_path = None; + for action in journal.actions.iter().rev() { + let current = inspect_file(&journal.source_root, &action.path)?; + if current == action.before { + continue; + } + if current != action.after { + foreign_path.get_or_insert_with(|| journal.source_root.join(&action.path)); + continue; + } + set_file(transaction_root, journal, action, Direction::Backward)?; + } + sync_directory(&journal.source_root)?; + if let Some(path) = foreign_path { + Err(PublicationError::ForeignContent { path }) + } else { + Ok(()) + } +} + +#[cfg(windows)] +fn reconcile_windows_scratch( + journal: &PublicationJournal, + direction: Direction, +) -> Result<(), PublicationError> { + use std::ffi::OsStr; + + use crate::support::windows_fs; + + for action in &journal.actions { + let Some(parent) = windows_fs::open_parent(&journal.source_root, &action.path, false) + .map_err(|source| { + io_error( + "open managed parent for recovery", + &journal.source_root.join(&action.path), + source, + ) + })? + else { + continue; + }; + let display = journal.source_root.join(&action.path); + let claim_name = OsStr::new(&action.claim); + let install_name = format!("{}.new", action.claim); + let install_name = OsStr::new(&install_name); + let target = windows_fs::read_optional(&parent) + .map_err(|source| io_error("inspect managed target for recovery", &display, source))?; + let claim = windows_fs::read_named_optional(&parent.directory, claim_name) + .map_err(|source| io_error("inspect managed claim for recovery", &display, source))?; + let install = + windows_fs::read_named_optional(&parent.directory, install_name).map_err(|source| { + io_error( + "inspect managed install file for recovery", + &display, + source, + ) + })?; + let known = |bytes: &[u8]| { + let hash = Some(hash_hex(bytes)); + hash == action.before.sha256 || hash == action.after.sha256 + }; + let desired_hash = match direction { + Direction::Forward => &action.after.sha256, + Direction::Backward => &action.before.sha256, + }; + if claim.as_deref().is_some_and(|bytes| !known(bytes)) { + return Err(PublicationError::ForeignContent { path: display }); + } + if install.as_deref().is_some_and(|bytes| !known(bytes)) { + return Err(PublicationError::ForeignContent { path: display }); + } + if let Some(target) = target { + if !known(&target) { + return Err(PublicationError::ForeignContent { path: display }); + } + remove_windows_scratch(&parent.directory, claim_name, &display)?; + remove_windows_scratch(&parent.directory, install_name, &display)?; + continue; + } + match install { + Some(bytes) + if desired_hash + .as_ref() + .is_some_and(|hash| hash == &hash_hex(&bytes)) => + { + let install = + windows_fs::open_named_existing(&parent.directory, install_name, true) + .map_err(|source| { + io_error("open managed install file for recovery", &display, source) + })?; + windows_fs::rename_to(&install, &parent.directory, &parent.file_name).map_err( + |source| io_error("finish managed install recovery", &display, source), + )?; + remove_windows_scratch(&parent.directory, claim_name, &display)?; + } + Some(_) => { + remove_windows_scratch(&parent.directory, install_name, &display)?; + if claim.is_some() { + let claim = + windows_fs::open_named_existing(&parent.directory, claim_name, true) + .map_err(|source| { + io_error("open managed claim for recovery", &display, source) + })?; + windows_fs::rename_to(&claim, &parent.directory, &parent.file_name).map_err( + |source| { + io_error("restore managed claim during recovery", &display, source) + }, + )?; + } + } + None if claim.is_some() => { + let claim = windows_fs::open_named_existing(&parent.directory, claim_name, true) + .map_err(|source| { + io_error("open managed claim for recovery", &display, source) + })?; + windows_fs::rename_to(&claim, &parent.directory, &parent.file_name).map_err( + |source| io_error("restore managed claim during recovery", &display, source), + )?; + } + None => {} + } + windows_fs::flush(&parent.directory) + .map_err(|source| io_error("sync managed parent after recovery", &display, source))?; + } + Ok(()) +} + +#[cfg(windows)] +fn remove_windows_scratch( + parent: &File, + name: &std::ffi::OsStr, + display: &Path, +) -> Result<(), PublicationError> { + use crate::support::windows_fs; + + match windows_fs::open_named_existing(parent, name, true) { + Ok(file) => windows_fs::delete_on_close(&file) + .map_err(|source| io_error("remove managed recovery scratch", display, source)), + Err(source) if source.kind() == io::ErrorKind::NotFound => Ok(()), + Err(source) => Err(io_error("open managed recovery scratch", display, source)), + } +} + +#[derive(Clone, Copy)] +enum Direction { + Forward, + Backward, +} + +fn set_file( + transaction_root: &Path, + journal: &PublicationJournal, + action: &JournalAction, + direction: Direction, +) -> Result<(), PublicationError> { + #[cfg(unix)] + return set_file_unix(transaction_root, journal, action, direction); + #[cfg(windows)] + return set_file_windows(transaction_root, journal, action, direction); + #[cfg(not(any(unix, windows)))] + Err(PublicationError::UnsupportedSafeFilesystem) +} + +#[cfg(windows)] +fn set_file_windows( + transaction_root: &Path, + journal: &PublicationJournal, + action: &JournalAction, + direction: Direction, +) -> Result<(), PublicationError> { + use std::ffi::OsStr; + + use crate::support::windows_fs; + + let parent = windows_fs::open_parent(&journal.source_root, &action.path, true) + .map_err(|source| { + io_error( + "open managed parent by handle", + &journal.source_root.join(&action.path), + source, + ) + })? + .ok_or_else(|| PublicationError::InvalidPath { + path: action.path.clone(), + })?; + #[cfg(test)] + AFTER_SAFE_PARENT_OPEN.with(|hook| { + if let Some(hook) = hook.borrow_mut().take() { + hook(); + } + }); + let rebound = + windows_fs::open_parent(&journal.source_root, &action.path, false).map_err(|source| { + io_error( + "reopen managed parent by handle", + &journal.source_root.join(&action.path), + source, + ) + })?; + if !rebound.as_ref().is_some_and(|rebound| { + windows_fs::same_directory(&parent.directory, &rebound.directory).unwrap_or(false) + }) { + return Err(PublicationError::ForeignContent { + path: journal.source_root.join(&action.path), + }); + } + + let (expected, current, blob) = match direction { + Direction::Forward => ( + &action.after, + &action.before, + action + .payload + .as_deref() + .map(|name| transaction_root.join("payloads").join(name)), + ), + Direction::Backward => ( + &action.before, + &action.after, + action + .backup + .as_deref() + .map(|name| transaction_root.join("backups").join(name)), + ), + }; + let display = journal.source_root.join(&action.path); + let claim_name = OsStr::new(&action.claim); + let install_name = format!("{}.new", action.claim); + let install_name = OsStr::new(&install_name); + let expected_bytes = match (&expected.sha256, blob) { + (None, None) => None, + (Some(expected_hash), Some(blob)) => { + let bytes = read_regular_no_follow(&blob)?; + if hash_hex(&bytes) != *expected_hash { + return Err(PublicationError::InvalidJournal { + path: blob, + reason: "payload hash mismatch".to_owned(), + }); + } + Some(bytes) + } + _ => { + return Err(PublicationError::InvalidJournal { + path: transaction_root.join(JOURNAL_FILE), + reason: "file state and payload disagree".to_owned(), + }) + } + }; + + // Deterministic transaction-owned scratch names may survive a process crash. A claim must + // always contain one of the journaled versions; an install file is safe to discard because it + // is never considered published until its handle-relative rename succeeds. + if let Some(bytes) = windows_fs::read_named_optional(&parent.directory, claim_name) + .map_err(|source| io_error("inspect managed publication claim", &display, source))? + { + let claim_hash = Some(hash_hex(&bytes)); + if claim_hash != action.before.sha256 && claim_hash != action.after.sha256 { + return Err(PublicationError::ForeignContent { path: display }); + } + let claim = windows_fs::open_named_existing(&parent.directory, claim_name, true) + .map_err(|source| io_error("open recoverable publication claim", &display, source))?; + windows_fs::delete_on_close(&claim) + .map_err(|source| io_error("remove recoverable publication claim", &display, source))?; + } + if let Some(bytes) = windows_fs::read_named_optional(&parent.directory, install_name) + .map_err(|source| io_error("inspect managed publication install file", &display, source))? + { + let install_hash = Some(hash_hex(&bytes)); + if install_hash != action.before.sha256 && install_hash != action.after.sha256 { + return Err(PublicationError::ForeignContent { path: display }); + } + let install = windows_fs::open_named_existing(&parent.directory, install_name, true) + .map_err(|source| { + io_error( + "open recoverable publication install file", + &display, + source, + ) + })?; + windows_fs::delete_on_close(&install).map_err(|source| { + io_error( + "remove recoverable publication install file", + &display, + source, + ) + })?; + } + + let target = match ¤t.sha256 { + Some(current_hash) => { + let mut target = windows_fs::open_regular_existing(&parent, true) + .map_err(|source| io_error("open managed file by handle", &display, source))?; + let actual = windows_fs::read_all(&mut target) + .map_err(|source| io_error("verify managed file by handle", &display, source))?; + if hash_hex(&actual) != *current_hash { + return Err(PublicationError::ForeignContent { path: display }); + } + windows_fs::rename_to(&target, &parent.directory, claim_name) + .map_err(|source| io_error("claim managed file by handle", &display, source))?; + Some(target) + } + None => None, + }; + + let result = (|| -> Result<(), PublicationError> { + match expected_bytes { + None => Ok(()), + Some(bytes) => { + let install = windows_fs::create_named(&parent.directory, install_name).map_err( + |source| io_error("create managed publication install file", &display, source), + )?; + let install = windows_fs::write_synced(install, &bytes).map_err(|source| { + io_error("write managed publication install file", &display, source) + })?; + windows_fs::rename_to(&install, &parent.directory, &parent.file_name) + .map_err(|source| io_error("publish managed file by handle", &display, source)) + } + } + })(); + + if let Err(error) = result { + if let Some(target) = target.as_ref() { + let _ = windows_fs::rename_to(target, &parent.directory, &parent.file_name); + } + return Err(error); + } + if let Some(target) = target.as_ref() { + windows_fs::delete_on_close(target) + .map_err(|source| io_error("remove replaced managed file", &display, source))?; + } + windows_fs::flush(&parent.directory) + .map_err(|source| io_error("sync managed parent directory", &display, source)) +} + +#[cfg(unix)] +struct ManagedParent { + directory: File, + file_name: std::ffi::CString, +} + +#[cfg(unix)] +fn set_file_unix( + transaction_root: &Path, + journal: &PublicationJournal, + action: &JournalAction, + direction: Direction, +) -> Result<(), PublicationError> { + use std::os::fd::AsRawFd; + + let parent = + open_managed_parent_unix(&journal.source_root, &action.path, true)?.ok_or_else(|| { + PublicationError::InvalidPath { + path: action.path.clone(), + } + })?; + #[cfg(test)] + AFTER_SAFE_PARENT_OPEN.with(|hook| { + if let Some(hook) = hook.borrow_mut().take() { + hook(); + } + }); + let rebound = open_managed_parent_unix(&journal.source_root, &action.path, false)?; + if !rebound + .as_ref() + .is_some_and(|rebound| same_directory(&parent.directory, &rebound.directory)) + { + return Err(PublicationError::ForeignContent { + path: journal.source_root.join(&action.path), + }); + } + + let (expected, current, blob) = match direction { + Direction::Forward => ( + &action.after, + &action.before, + action + .payload + .as_deref() + .map(|name| transaction_root.join("payloads").join(name)), + ), + Direction::Backward => ( + &action.before, + &action.after, + action + .backup + .as_deref() + .map(|name| transaction_root.join("backups").join(name)), + ), + }; + let claim = c_string_component(&action.claim, &action.path)?; + clean_known_claim_unix(&parent.directory, &claim, action)?; + match (&expected.sha256, blob) { + (None, None) => remove_regular_claimed_unix( + &parent.directory, + &parent.file_name, + current, + &claim, + &journal.source_root.join(&action.path), + )?, + (Some(hash), Some(blob)) => replace_from_blob_unix( + &parent.directory, + &parent.file_name, + &blob, + hash, + current, + &claim, + &journal.source_root.join(&action.path), + )?, + _ => { + return Err(PublicationError::InvalidJournal { + path: transaction_root.join(JOURNAL_FILE), + reason: "file state and payload disagree".to_owned(), + }) + } + } + parent.directory.sync_all().map_err(|source| { + io_error( + "sync managed parent directory", + &journal.source_root, + source, + ) + })?; + let _ = parent.directory.as_raw_fd(); + Ok(()) +} + +#[cfg(unix)] +fn open_managed_parent_unix( + root: &Path, + relative: &str, + create: bool, +) -> Result, PublicationError> { + use std::os::fd::{AsRawFd, FromRawFd}; + use std::os::unix::fs::OpenOptionsExt; + + validate_relative(relative)?; + let mut options = OpenOptions::new(); + options + .read(true) + .custom_flags(libc::O_DIRECTORY | libc::O_NOFOLLOW | libc::O_CLOEXEC); + let mut directory = options + .open(root) + .map_err(|source| io_error("open managed root without following links", root, source))?; + let components = Path::new(relative).components().collect::>(); + let Some((file_component, parents)) = components.split_last() else { + return Err(PublicationError::InvalidPath { + path: relative.to_owned(), + }); + }; + for component in parents { + let Component::Normal(name) = component else { + return Err(PublicationError::InvalidPath { + path: relative.to_owned(), + }); + }; + let name = c_string_os_component(name, relative)?; + let mut descriptor = unsafe { + libc::openat( + directory.as_raw_fd(), + name.as_ptr(), + libc::O_RDONLY | libc::O_DIRECTORY | libc::O_NOFOLLOW | libc::O_CLOEXEC, + ) + }; + if descriptor < 0 { + let error = io::Error::last_os_error(); + if error.raw_os_error() == Some(libc::ELOOP) + || error.kind() == io::ErrorKind::NotADirectory + { + return Err(PublicationError::UnsafeFile { + path: root.join(relative), + }); + } + if error.kind() == io::ErrorKind::NotFound && !create { + return Ok(None); + } + if error.kind() != io::ErrorKind::NotFound || !create { + return Err(io_error( + "open managed parent without following links", + root, + error, + )); + } + let created = unsafe { libc::mkdirat(directory.as_raw_fd(), name.as_ptr(), 0o755) }; + if created != 0 { + let source = io::Error::last_os_error(); + if source.kind() != io::ErrorKind::AlreadyExists { + return Err(io_error("create managed parent directory", root, source)); + } + } + directory + .sync_all() + .map_err(|source| io_error("sync managed parent directory", root, source))?; + descriptor = unsafe { + libc::openat( + directory.as_raw_fd(), + name.as_ptr(), + libc::O_RDONLY | libc::O_DIRECTORY | libc::O_NOFOLLOW | libc::O_CLOEXEC, + ) + }; + if descriptor < 0 { + return Err(io_error( + "open created managed parent directory", + root, + io::Error::last_os_error(), + )); + } + } + directory = unsafe { File::from_raw_fd(descriptor) }; + } + let Component::Normal(file_name) = file_component else { + return Err(PublicationError::InvalidPath { + path: relative.to_owned(), + }); + }; + Ok(Some(ManagedParent { + directory, + file_name: c_string_os_component(file_name, relative)?, + })) +} + +#[cfg(unix)] +fn same_directory(left: &File, right: &File) -> bool { + use std::os::unix::fs::MetadataExt; + match (left.metadata(), right.metadata()) { + (Ok(left), Ok(right)) => left.dev() == right.dev() && left.ino() == right.ino(), + (Err(_), _) | (_, Err(_)) => false, + } +} + +#[cfg(unix)] +fn c_string_os_component( + value: &std::ffi::OsStr, + relative: &str, +) -> Result { + use std::os::unix::ffi::OsStrExt; + std::ffi::CString::new(value.as_bytes()).map_err(|_| PublicationError::InvalidPath { + path: relative.to_owned(), + }) +} + +#[cfg(unix)] +fn c_string_component(value: &str, relative: &str) -> Result { + std::ffi::CString::new(value).map_err(|_| PublicationError::InvalidPath { + path: relative.to_owned(), + }) +} + +#[cfg(unix)] +fn read_at_optional( + directory: &File, + name: &std::ffi::CStr, + display: &Path, +) -> Result>, PublicationError> { + use std::os::fd::{AsRawFd, FromRawFd}; + let descriptor = unsafe { + libc::openat( + directory.as_raw_fd(), + name.as_ptr(), + libc::O_RDONLY | libc::O_NOFOLLOW | libc::O_CLOEXEC, + ) + }; + if descriptor < 0 { + let source = io::Error::last_os_error(); + return if source.kind() == io::ErrorKind::NotFound { + Ok(None) + } else { + Err(io_error( + "open managed file without following links", + display, + source, + )) + }; + } + let mut file = unsafe { File::from_raw_fd(descriptor) }; + if !file + .metadata() + .map_err(|source| io_error("inspect managed file", display, source))? + .file_type() + .is_file() + { + return Err(PublicationError::UnsafeFile { + path: display.to_path_buf(), + }); + } + let mut bytes = Vec::new(); + file.read_to_end(&mut bytes) + .map_err(|source| io_error("read managed file", display, source))?; + Ok(Some(bytes)) +} + +#[cfg(unix)] +fn unlink_at( + directory: &File, + name: &std::ffi::CStr, + flags: libc::c_int, + display: &Path, +) -> Result<(), PublicationError> { + use std::os::fd::AsRawFd; + if unsafe { libc::unlinkat(directory.as_raw_fd(), name.as_ptr(), flags) } == 0 { + Ok(()) + } else { + Err(io_error( + "remove managed entry", + display, + io::Error::last_os_error(), + )) + } +} + +#[cfg(unix)] +fn link_at( + directory: &File, + source: &std::ffi::CStr, + target: &std::ffi::CStr, + display: &Path, +) -> Result<(), PublicationError> { + use std::os::fd::AsRawFd; + let result = unsafe { + libc::linkat( + directory.as_raw_fd(), + source.as_ptr(), + directory.as_raw_fd(), + target.as_ptr(), + 0, + ) + }; + if result == 0 { + Ok(()) + } else { + let source = io::Error::last_os_error(); + if source.kind() == io::ErrorKind::AlreadyExists { + Err(PublicationError::ForeignContent { + path: display.to_path_buf(), + }) + } else { + Err(io_error("link managed entry", display, source)) + } + } +} + +#[cfg(unix)] +fn clean_known_claim_unix( + directory: &File, + claim: &std::ffi::CStr, + action: &JournalAction, +) -> Result<(), PublicationError> { + let display = Path::new(&action.claim); + if let Some(bytes) = read_at_optional(directory, claim, display)? { + let hash = Some(hash_hex(&bytes)); + if hash != action.before.sha256 && hash != action.after.sha256 { + return Err(PublicationError::ForeignContent { + path: display.to_path_buf(), + }); + } + unlink_at(directory, claim, 0, display)?; + directory + .sync_all() + .map_err(|source| io_error("sync managed parent directory", display, source))?; + } + Ok(()) +} + +#[cfg(unix)] +fn remove_regular_claimed_unix( + directory: &File, + target: &std::ffi::CStr, + current: &ExpectedFile, + claim: &std::ffi::CStr, + display: &Path, +) -> Result<(), PublicationError> { + let Some(current_hash) = ¤t.sha256 else { + return Ok(()); + }; + rename_at_no_replace(directory, target, claim, display)?; + let claimed = read_at_optional(directory, claim, display)?.ok_or_else(|| { + PublicationError::ForeignContent { + path: display.to_path_buf(), + } + })?; + if hash_hex(&claimed) != *current_hash { + rename_at_no_replace(directory, claim, target, display)?; + return Err(PublicationError::ForeignContent { + path: display.to_path_buf(), + }); + } + unlink_at(directory, claim, 0, display) +} + +#[cfg(target_os = "macos")] +fn rename_at_no_replace( + directory: &File, + left: &std::ffi::CStr, + right: &std::ffi::CStr, + display: &Path, +) -> Result<(), PublicationError> { + rename_at_with_flags(directory, left, right, libc::RENAME_EXCL, display) +} + +#[cfg(target_os = "linux")] +fn rename_at_no_replace( + directory: &File, + left: &std::ffi::CStr, + right: &std::ffi::CStr, + display: &Path, +) -> Result<(), PublicationError> { + rename_at_with_flags(directory, left, right, libc::RENAME_NOREPLACE, display) +} + +#[cfg(target_os = "macos")] +fn exchange_at( + directory: &File, + left: &std::ffi::CStr, + right: &std::ffi::CStr, + display: &Path, +) -> Result<(), PublicationError> { + rename_at_with_flags(directory, left, right, libc::RENAME_SWAP, display) +} + +#[cfg(target_os = "linux")] +fn exchange_at( + directory: &File, + left: &std::ffi::CStr, + right: &std::ffi::CStr, + display: &Path, +) -> Result<(), PublicationError> { + rename_at_with_flags(directory, left, right, libc::RENAME_EXCHANGE, display) +} + +#[cfg(target_os = "macos")] +fn rename_at_with_flags( + directory: &File, + left: &std::ffi::CStr, + right: &std::ffi::CStr, + flags: libc::c_uint, + display: &Path, +) -> Result<(), PublicationError> { + use std::os::fd::AsRawFd; + let result = unsafe { + libc::renameatx_np( + directory.as_raw_fd(), + left.as_ptr(), + directory.as_raw_fd(), + right.as_ptr(), + flags, + ) + }; + classify_rename_at(result, display) +} + +#[cfg(target_os = "linux")] +fn rename_at_with_flags( + directory: &File, + left: &std::ffi::CStr, + right: &std::ffi::CStr, + flags: libc::c_uint, + display: &Path, +) -> Result<(), PublicationError> { + use std::os::fd::AsRawFd; + let result = unsafe { + libc::renameat2( + directory.as_raw_fd(), + left.as_ptr(), + directory.as_raw_fd(), + right.as_ptr(), + flags, + ) + }; + classify_rename_at(result, display) +} + +#[cfg(unix)] +fn classify_rename_at(result: libc::c_int, display: &Path) -> Result<(), PublicationError> { + if result == 0 { + Ok(()) + } else { + let source = io::Error::last_os_error(); + if matches!( + source.kind(), + io::ErrorKind::AlreadyExists | io::ErrorKind::NotFound + ) { + Err(PublicationError::ForeignContent { + path: display.to_path_buf(), + }) + } else { + Err(io_error("atomically claim managed file", display, source)) + } + } +} + +#[cfg(unix)] +fn replace_from_blob_unix( + directory: &File, + target: &std::ffi::CStr, + blob: &Path, + expected_hash: &str, + current: &ExpectedFile, + claim: &std::ffi::CStr, + display: &Path, +) -> Result<(), PublicationError> { + use std::os::fd::{AsRawFd, FromRawFd}; + let bytes = read_regular_no_follow(blob)?; + if hash_hex(&bytes) != expected_hash { + return Err(PublicationError::InvalidJournal { + path: blob.to_path_buf(), + reason: "payload hash mismatch".to_owned(), + }); + } + let descriptor = unsafe { + libc::openat( + directory.as_raw_fd(), + claim.as_ptr(), + libc::O_WRONLY | libc::O_CREAT | libc::O_EXCL | libc::O_NOFOLLOW | libc::O_CLOEXEC, + 0o600, + ) + }; + if descriptor < 0 { + return Err(io_error( + "create managed publication claim", + display, + io::Error::last_os_error(), + )); + } + let mut output = unsafe { File::from_raw_fd(descriptor) }; + output + .write_all(&bytes) + .map_err(|source| io_error("write managed publication claim", display, source))?; + output + .sync_all() + .map_err(|source| io_error("sync managed publication claim", display, source))?; + drop(output); + match ¤t.sha256 { + None => { + link_at(directory, claim, target, display)?; + unlink_at(directory, claim, 0, display)?; + } + Some(current_hash) => { + exchange_at(directory, claim, target, display)?; + let swapped = read_at_optional(directory, claim, display)?.ok_or_else(|| { + PublicationError::ForeignContent { + path: display.to_path_buf(), + } + })?; + if hash_hex(&swapped) != *current_hash { + exchange_at(directory, claim, target, display)?; + let _ = unlink_at(directory, claim, 0, display); + return Err(PublicationError::ForeignContent { + path: display.to_path_buf(), + }); + } + unlink_at(directory, claim, 0, display)?; + } + } + Ok(()) +} + +#[cfg(not(any(unix, windows)))] +fn replace_from_blob( + target: &Path, + blob: &Path, + expected_hash: &str, + current: &ExpectedFile, + claim_name: &str, +) -> Result<(), PublicationError> { + let bytes = read_regular_no_follow(blob)?; + if hash_hex(&bytes) != expected_hash { + return Err(PublicationError::InvalidJournal { + path: blob.to_path_buf(), + reason: "payload hash mismatch".to_owned(), + }); + } + let parent = target + .parent() + .ok_or_else(|| PublicationError::InvalidPath { + path: target.display().to_string(), + })?; + let temp = parent.join(claim_name); + let mut options = OpenOptions::new(); + options.write(true).create_new(true); + #[cfg(unix)] + { + use std::os::unix::fs::OpenOptionsExt; + options.mode(0o600).custom_flags(libc::O_NOFOLLOW); + } + let mut file = options + .open(&temp) + .map_err(|source| io_error("create publication file", &temp, source))?; + file.write_all(&bytes) + .map_err(|source| io_error("write publication file", &temp, source))?; + file.sync_all() + .map_err(|source| io_error("sync publication file", &temp, source))?; + drop(file); + match ¤t.sha256 { + None => { + fs::hard_link(&temp, target).map_err(|source| { + if source.kind() == io::ErrorKind::AlreadyExists { + PublicationError::ForeignContent { + path: target.to_path_buf(), + } + } else { + io_error("claim absent managed path", target, source) + } + })?; + fs::remove_file(&temp) + .map_err(|source| io_error("remove publication temporary file", &temp, source))?; + } + Some(current_hash) => { + exchange_paths(&temp, target)?; + let swapped = read_regular_no_follow(&temp)?; + if hash_hex(&swapped) != *current_hash { + exchange_paths(&temp, target)?; + let _ = fs::remove_file(&temp); + return Err(PublicationError::ForeignContent { + path: target.to_path_buf(), + }); + } + fs::remove_file(&temp) + .map_err(|source| io_error("remove replaced managed file", &temp, source))?; + } + } + Ok(()) +} + +#[cfg(not(any(unix, windows)))] +fn clean_known_claim(target: &Path, action: &JournalAction) -> Result<(), PublicationError> { + let parent = target + .parent() + .ok_or_else(|| PublicationError::InvalidPath { + path: target.display().to_string(), + })?; + let claim = parent.join(&action.claim); + match read_regular_no_follow(&claim) { + Ok(bytes) => { + let hash = Some(hash_hex(&bytes)); + if hash != action.before.sha256 && hash != action.after.sha256 { + return Err(PublicationError::ForeignContent { path: claim }); + } + fs::remove_file(&claim).map_err(|source| { + io_error("remove recoverable publication claim", &claim, source) + })?; + sync_directory(parent) + } + Err(PublicationError::Io { source, .. }) if source.kind() == io::ErrorKind::NotFound => { + Ok(()) + } + Err(error) => Err(error), + } +} + +#[cfg(not(any(unix, windows)))] +fn remove_regular_claimed( + path: &Path, + current: &ExpectedFile, + claim_name: &str, +) -> Result<(), PublicationError> { + let Some(current_hash) = ¤t.sha256 else { + return Ok(()); + }; + let parent = path.parent().ok_or_else(|| PublicationError::InvalidPath { + path: path.display().to_string(), + })?; + let claim = parent.join(claim_name); + rename_no_replace(path, &claim)?; + let claimed = read_regular_no_follow(&claim)?; + if hash_hex(&claimed) != *current_hash { + rename_no_replace(&claim, path)?; + return Err(PublicationError::ForeignContent { + path: path.to_path_buf(), + }); + } + fs::remove_file(&claim) + .map_err(|source| io_error("remove claimed managed file", &claim, source)) +} + +#[cfg(not(any(target_os = "macos", target_os = "linux", windows)))] +fn exchange_paths(left: &Path, right: &Path) -> Result<(), PublicationError> { + let claim = left.with_extension("exchange"); + fs::rename(right, &claim).map_err(|source| io_error("claim managed file", right, source))?; + if let Err(source) = fs::rename(left, right) { + let _ = fs::rename(&claim, right); + return Err(io_error("publish managed file", right, source)); + } + fs::rename(&claim, left).map_err(|source| io_error("finish managed exchange", left, source)) +} + +#[cfg(not(any(target_os = "macos", target_os = "linux", windows)))] +fn rename_no_replace(left: &Path, right: &Path) -> Result<(), PublicationError> { + if right + .try_exists() + .map_err(|source| io_error("inspect claim path", right, source))? + { + return Err(PublicationError::ForeignContent { + path: right.to_path_buf(), + }); + } + fs::rename(left, right).map_err(|source| io_error("claim managed file", left, source)) +} + +fn read_expected( + root: &Path, + relative: &str, + expected: FileVersion, +) -> Result>, PublicationError> { + let actual = read_optional(root, relative)?; + let actual_version = actual.as_deref().map_or(FileVersion::Absent, |bytes| { + FileVersion::Present(hash_raw(bytes)) + }); + if actual_version != expected { + return Err(PublicationError::SnapshotMismatch { + path: relative.to_owned(), + }); + } + Ok(actual) +} + +fn read_optional(root: &Path, relative: &str) -> Result>, PublicationError> { + #[cfg(unix)] + { + let Some(parent) = open_managed_parent_unix(root, relative, false)? else { + return Ok(None); + }; + return read_at_optional(&parent.directory, &parent.file_name, &root.join(relative)); + } + #[cfg(windows)] + { + validate_relative(relative)?; + let Some(parent) = + crate::support::windows_fs::open_parent(root, relative, false).map_err(|source| { + io_error( + "open managed parent by handle", + &root.join(relative), + source, + ) + })? + else { + return Ok(None); + }; + return crate::support::windows_fs::read_optional(&parent).map_err(|source| { + io_error("read managed file by handle", &root.join(relative), source) + }); + } + #[cfg(not(any(unix, windows)))] + Err(PublicationError::UnsupportedSafeFilesystem) +} + +fn read_regular_no_follow(path: &Path) -> Result, PublicationError> { + let mut options = OpenOptions::new(); + options.read(true); + #[cfg(unix)] + { + use std::os::unix::fs::OpenOptionsExt; + options.custom_flags(libc::O_NOFOLLOW); + } + let mut file = options + .open(path) + .map_err(|source| io_error("open regular file without following links", path, source))?; + let metadata = file + .metadata() + .map_err(|source| io_error("inspect opened file", path, source))?; + if !metadata.file_type().is_file() { + return Err(PublicationError::UnsafeFile { + path: path.to_path_buf(), + }); + } + let mut bytes = Vec::new(); + file.read_to_end(&mut bytes) + .map_err(|source| io_error("read regular file", path, source))?; + Ok(bytes) +} + +fn inspect_file(root: &Path, relative: &str) -> Result { + Ok(expected(read_optional(root, relative)?.as_deref())) +} + +fn expected(bytes: Option<&[u8]>) -> ExpectedFile { + ExpectedFile { + sha256: bytes.map(hash_hex), + } +} +fn hash_raw(bytes: &[u8]) -> RawFileHash { + Sha256::digest(bytes).into() +} +fn hash_hex(bytes: &[u8]) -> String { + format!("{:x}", Sha256::digest(bytes)) +} + +fn write_blob( + root: &Path, + index: usize, + bytes: Option<&[u8]>, +) -> Result, PublicationError> { + let Some(bytes) = bytes else { + return Ok(None); + }; + let name = format!("{index:08}.bin"); + let path = root.join(&name); + let mut file = OpenOptions::new() + .write(true) + .create_new(true) + .open(&path) + .map_err(|source| io_error("create transaction blob", &path, source))?; + file.write_all(bytes) + .map_err(|source| io_error("write transaction blob", &path, source))?; + file.sync_all() + .map_err(|source| io_error("sync transaction blob", &path, source))?; + Ok(Some(name)) +} + +fn write_journal(root: &Path, journal: &PublicationJournal) -> Result<(), PublicationError> { + #[cfg(test)] + if FAIL_JOURNAL_STATUS.with(|slot| { + let should_fail = slot.get() == Some(journal.status); + if should_fail { + slot.set(None); + } + should_fail + }) { + return Err(io_error( + "write injected publication journal", + &root.join(JOURNAL_FILE), + io::Error::other("injected journal failure"), + )); + } + let path = root.join(JOURNAL_FILE); + let temp = root.join("journal.next"); + let bytes = + serde_json::to_vec_pretty(journal).map_err(|error| PublicationError::InvalidJournal { + path: path.clone(), + reason: error.to_string(), + })?; + let mut file = OpenOptions::new() + .write(true) + .create(true) + .truncate(true) + .open(&temp) + .map_err(|source| io_error("create publication journal", &temp, source))?; + file.write_all(&bytes) + .map_err(|source| io_error("write publication journal", &temp, source))?; + file.sync_all() + .map_err(|source| io_error("sync publication journal", &temp, source))?; + drop(file); + fs::rename(&temp, &path) + .map_err(|source| io_error("publish publication journal", &path, source))?; + sync_directory(root) +} + +fn read_journal(root: &Path) -> Result { + let path = root.join(JOURNAL_FILE); + let bytes = read_regular_no_follow(&path)?; + let journal = + serde_json::from_slice(&bytes).map_err(|error| PublicationError::InvalidJournal { + path, + reason: error.to_string(), + })?; + validate_journal(root, &journal)?; + Ok(journal) +} + +fn validate_journal(root: &Path, journal: &PublicationJournal) -> Result<(), PublicationError> { + let invalid = |reason: &str| PublicationError::InvalidJournal { + path: root.join(JOURNAL_FILE), + reason: reason.to_owned(), + }; + if journal.generation == 0 { + return Err(invalid("generation must be positive")); + } + if journal.target_identity.0.is_empty() || !journal.source_root.is_absolute() { + return Err(invalid( + "target identity and absolute source root are required", + )); + } + let mut previous_path: Option<&str> = None; + let private_root_component = root + .strip_prefix(&journal.source_root) + .ok() + .and_then(|relative| relative.components().next()) + .and_then(|component| match component { + Component::Normal(name) => Some(name), + Component::Prefix(_) + | Component::RootDir + | Component::CurDir + | Component::ParentDir => None, + }); + for action in &journal.actions { + validate_relative(&action.path)?; + if is_ignored_journal_path(&action.path, private_root_component) { + return Err(invalid( + "action path belongs to ignored or private inventory", + )); + } + if previous_path.is_some_and(|previous| previous >= action.path.as_str()) { + return Err(invalid("action paths must be strictly sorted and unique")); + } + previous_path = Some(&action.path); + validate_expected_hash(&action.before, "before", &invalid)?; + validate_expected_hash(&action.after, "after", &invalid)?; + validate_blob_name( + action.backup.as_deref(), + action.before.sha256.is_some(), + &invalid, + )?; + validate_blob_name( + action.payload.as_deref(), + action.after.sha256.is_some(), + &invalid, + )?; + if !is_single_normal_component(&action.claim) + || !action.claim.starts_with(".v8-runner-publication-claim-") + { + return Err(invalid("claim name must be one owned normal component")); + } + } + Ok(()) +} + +fn is_ignored_journal_path(path: &str, private_root: Option<&std::ffi::OsStr>) -> bool { + is_always_ignored_relative_path(Path::new(path)) + || Path::new(path) + .components() + .any(|component| match component { + Component::Normal(name) => { + let text = name.to_string_lossy(); + text.starts_with(".v8-runner-publication-claim-") + || private_root.is_some_and(|private| name == private) + } + Component::Prefix(_) + | Component::RootDir + | Component::CurDir + | Component::ParentDir => true, + }) +} + +fn validate_expected_hash( + value: &ExpectedFile, + label: &str, + invalid: &impl Fn(&str) -> PublicationError, +) -> Result<(), PublicationError> { + if value + .sha256 + .as_ref() + .is_some_and(|hash| hash.len() != 64 || !hash.bytes().all(|byte| byte.is_ascii_hexdigit())) + { + return Err(invalid(&format!("{label} hash must be a raw SHA-256"))); + } + Ok(()) +} + +fn validate_blob_name( + name: Option<&str>, + required: bool, + invalid: &impl Fn(&str) -> PublicationError, +) -> Result<(), PublicationError> { + if name.is_some() != required { + return Err(invalid( + "blob presence must match the corresponding file state", + )); + } + if name.is_some_and(|name| !is_single_normal_component(name)) { + return Err(invalid("blob name must be one normal path component")); + } + Ok(()) +} + +fn is_single_normal_component(value: &str) -> bool { + let mut components = Path::new(value).components(); + matches!(components.next(), Some(Component::Normal(_))) && components.next().is_none() +} + +fn cleanup_transaction(root: &Path) -> Result<(), PublicationError> { + let parent = root.parent().map(Path::to_path_buf); + fs::remove_dir_all(root) + .map_err(|source| io_error("remove committed transaction", root, source))?; + if let Some(parent) = parent { + sync_directory(&parent)?; + } + Ok(()) +} + +fn canonical_directory(path: &Path) -> Result { + let canonical = fs::canonicalize(path) + .map_err(|source| io_error("canonicalize directory", path, source))?; + let metadata = fs::symlink_metadata(&canonical) + .map_err(|source| io_error("inspect canonical directory", &canonical, source))?; + if !metadata.file_type().is_dir() || metadata.file_type().is_symlink() { + return Err(PublicationError::UnsafeFile { path: canonical }); + } + Ok(canonical) +} + +fn validate_relative(path: &str) -> Result<(), PublicationError> { + let candidate = Path::new(path); + if path.is_empty() + || path.contains('\\') + || path.eq_ignore_ascii_case("ConfigDumpInfo.xml") + || candidate + .components() + .any(|component| !matches!(component, Component::Normal(_))) + { + return Err(PublicationError::InvalidPath { + path: path.to_owned(), + }); + } + Ok(()) +} + +#[cfg(not(any(unix, windows)))] +fn ensure_safe_parents(root: &Path, relative: &str, create: bool) -> Result<(), PublicationError> { + let mut current = root.to_path_buf(); + let parent = Path::new(relative) + .parent() + .unwrap_or_else(|| Path::new("")); + for component in parent.components() { + let Component::Normal(name) = component else { + return Err(PublicationError::InvalidPath { + path: relative.to_owned(), + }); + }; + current.push(name); + match fs::symlink_metadata(¤t) { + Ok(metadata) if metadata.file_type().is_dir() && !metadata.file_type().is_symlink() => { + } + Ok(_) => return Err(PublicationError::UnsafeFile { path: current }), + Err(error) if error.kind() == io::ErrorKind::NotFound && create => { + fs::create_dir(¤t).map_err(|source| { + io_error("create managed parent directory", ¤t, source) + })?; + if let Some(parent) = current.parent() { + sync_directory(parent)?; + } + } + Err(error) if error.kind() == io::ErrorKind::NotFound => break, + Err(source) => { + return Err(io_error( + "inspect managed parent directory", + ¤t, + source, + )) + } + } + } + Ok(()) +} + +fn sync_directory(path: &Path) -> Result<(), PublicationError> { + #[cfg(windows)] + { + let directory = crate::support::windows_fs::open_root(path) + .map_err(|source| io_error("open directory for sync", path, source))?; + return crate::support::windows_fs::flush(&directory) + .map_err(|source| io_error("sync directory", path, source)); + } + #[cfg(not(windows))] + File::open(path) + .and_then(|directory| directory.sync_all()) + .map_err(|source| io_error("sync directory", path, source)) +} + +fn io_error(operation: &'static str, path: &Path, source: io::Error) -> PublicationError { + PublicationError::Io { + operation, + path: path.to_path_buf(), + source, + } +} + +#[cfg(test)] +mod tests { + use std::collections::BTreeMap; + use std::fs; + + use sha2::{Digest, Sha256}; + + use crate::use_cases::shadow_merge::plan_manifest_merge; + + use super::{ + recover_publication, DumpTransactionId, ObservedStateGeneration, PublicationError, + PublicationRequest, TargetIdentity, AFTER_SAFE_PARENT_OPEN, BEFORE_ACTION_INSTALL, + BEFORE_FIRST_MUTATION, FAIL_JOURNAL_STATUS, + }; + + #[test] + fn conflict_rejects_the_whole_plan_before_any_write() { + let fixture = Fixture::new(); + fixture.write_source("a.txt", b"local"); + fixture.write_source("keep.txt", b"old"); + fixture.write_dump("a.txt", b"dump"); + fixture.write_dump("keep.txt", b"new"); + let baseline = manifest(&[("a.txt", b"base"), ("keep.txt", b"old")]); + let source = fixture.source_manifest(&["a.txt", "keep.txt"]); + let dump = fixture.dump_manifest(&["a.txt", "keep.txt"]); + let plan = plan_manifest_merge(&baseline, &source, &dump); + + let error = fixture.request(&plan).prepare().expect_err("conflict"); + + assert!(matches!(error, PublicationError::Conflict)); + assert_eq!(fs::read(fixture.source.join("keep.txt")).unwrap(), b"old"); + assert!(!fixture.transaction.exists()); + } + + #[test] + fn applies_only_manifest_paths_and_preserves_unmanaged_entries() { + let fixture = Fixture::new(); + fixture.write_source("update.txt", b"old"); + fixture.write_source("delete.txt", b"old-delete"); + fixture.write_source(".git/index", b"git"); + fixture.write_source("work/nested.bin", b"work"); + fixture.write_dump("update.txt", b"new"); + fixture.write_dump("create.txt", b"created"); + let baseline = manifest(&[("update.txt", b"old"), ("delete.txt", b"old-delete")]); + let source = fixture.source_manifest(&["update.txt", "delete.txt"]); + let dump = fixture.dump_manifest(&["update.txt", "create.txt"]); + let plan = plan_manifest_merge(&baseline, &source, &dump); + + fixture + .request(&plan) + .prepare() + .unwrap() + .apply() + .unwrap() + .mark_state_visible(fixture.observed(7)) + .unwrap() + .commit() + .unwrap(); + + assert_eq!(fs::read(fixture.source.join("update.txt")).unwrap(), b"new"); + assert_eq!( + fs::read(fixture.source.join("create.txt")).unwrap(), + b"created" + ); + assert!(!fixture.source.join("delete.txt").exists()); + assert_eq!(fs::read(fixture.source.join(".git/index")).unwrap(), b"git"); + assert_eq!( + fs::read(fixture.source.join("work/nested.bin")).unwrap(), + b"work" + ); + assert!(!fixture.transaction.exists()); + } + + #[test] + fn revalidates_every_source_version_before_the_first_mutation() { + let fixture = Fixture::new(); + fixture.write_source("a.txt", b"old-a"); + fixture.write_source("b.txt", b"old-b"); + fixture.write_dump("a.txt", b"new-a"); + fixture.write_dump("b.txt", b"new-b"); + let baseline = fixture.source_manifest(&["a.txt", "b.txt"]); + let source = baseline.clone(); + let dump = fixture.dump_manifest(&["a.txt", "b.txt"]); + let plan = plan_manifest_merge(&baseline, &source, &dump); + let prepared = fixture.request(&plan).prepare().unwrap(); + let changed = fixture.source.join("b.txt"); + BEFORE_FIRST_MUTATION.with(|hook| { + *hook.borrow_mut() = Some(Box::new(move || { + fs::write(changed, b"foreign").unwrap(); + })); + }); + + let error = prepared.apply().expect_err("TOCTOU"); + + assert!(matches!( + error, + PublicationError::PublicationAndRollback { .. } + )); + assert_eq!(fs::read(fixture.source.join("a.txt")).unwrap(), b"old-a"); + assert!(fixture.transaction.exists()); + } + + #[test] + fn create_does_not_clobber_a_file_installed_after_revalidation() { + let fixture = Fixture::new(); + fixture.write_dump("new.txt", b"dump"); + let plan = plan_manifest_merge( + &BTreeMap::new(), + &BTreeMap::new(), + &fixture.dump_manifest(&["new.txt"]), + ); + let prepared = fixture.request(&plan).prepare().unwrap(); + let foreign = fixture.source.join("new.txt"); + BEFORE_ACTION_INSTALL.with(|hook| { + *hook.borrow_mut() = Some(Box::new(move || { + fs::write(foreign, b"foreign").unwrap(); + })); + }); + + let error = prepared.apply().expect_err("foreign create"); + + assert!(matches!( + error, + PublicationError::PublicationAndRollback { .. } + )); + assert_eq!( + fs::read(fixture.source.join("new.txt")).unwrap(), + b"foreign" + ); + } + + #[test] + fn recovery_rolls_back_source_applied_and_finishes_state_visible() { + for state_visible in [false, true] { + let fixture = Fixture::new(); + fixture.write_source("file.txt", b"old"); + fixture.write_dump("file.txt", b"new"); + let baseline = fixture.source_manifest(&["file.txt"]); + let plan = + plan_manifest_merge(&baseline, &baseline, &fixture.dump_manifest(&["file.txt"])); + let applied = fixture.request(&plan).prepare().unwrap().apply().unwrap(); + if state_visible { + applied + .mark_state_visible(fixture.observed(7)) + .unwrap() + .leave_for_recovery(); + } else { + applied.leave_for_recovery(); + } + + recover_publication( + &fixture.transaction, + &fixture.source, + &TargetIdentity::new("ib-a"), + fixture.observed(if state_visible { 7 } else { 6 }), + ) + .unwrap(); + + let expected = if state_visible { + b"new".as_slice() + } else { + b"old".as_slice() + }; + assert_eq!(fs::read(fixture.source.join("file.txt")).unwrap(), expected); + assert!(!fixture.transaction.exists()); + } + } + + #[test] + fn recovery_is_deterministic_after_every_durable_phase() { + for phase in ["prepared", "source_applied", "state_visible", "committed"] { + let fixture = Fixture::new(); + fixture.write_source("file.txt", b"old"); + fixture.write_dump("file.txt", b"new"); + let baseline = fixture.source_manifest(&["file.txt"]); + let plan = + plan_manifest_merge(&baseline, &baseline, &fixture.dump_manifest(&["file.txt"])); + let prepared = fixture.request(&plan).prepare().unwrap(); + match phase { + "prepared" => prepared.leave_for_recovery(), + "source_applied" => prepared.apply().unwrap().leave_for_recovery(), + "state_visible" => prepared + .apply() + .unwrap() + .mark_state_visible(fixture.observed(7)) + .unwrap() + .leave_for_recovery(), + "committed" => prepared + .apply() + .unwrap() + .mark_state_visible(fixture.observed(7)) + .unwrap() + .mark_committed_for_recovery(), + _ => unreachable!(), + } + + let rolls_back = matches!(phase, "prepared" | "source_applied"); + recover_publication( + &fixture.transaction, + &fixture.source, + &TargetIdentity::new("ib-a"), + fixture.observed(if rolls_back { 6 } else { 7 }), + ) + .unwrap(); + + assert_eq!( + fs::read(fixture.source.join("file.txt")).unwrap(), + if rolls_back { b"old" } else { b"new" } + ); + } + } + + #[cfg(unix)] + #[test] + fn rejects_symlinked_managed_parent_without_touching_its_target() { + use std::os::unix::fs::symlink; + + let fixture = Fixture::new(); + let outside = fixture._temp.path().join("outside"); + fs::create_dir(&outside).unwrap(); + fs::write(outside.join("file.txt"), b"old").unwrap(); + symlink(&outside, fixture.source.join("linked")).unwrap(); + fixture.write_dump("linked/file.txt", b"dump"); + let plan = plan_manifest_merge( + &BTreeMap::new(), + &BTreeMap::new(), + &fixture.dump_manifest(&["linked/file.txt"]), + ); + + let error = fixture.request(&plan).prepare().expect_err("symlink"); + + assert!(matches!(error, PublicationError::UnsafeFile { .. })); + assert_eq!(fs::read(outside.join("file.txt")).unwrap(), b"old"); + } + + #[test] + fn recovery_never_clobbers_foreign_content() { + let fixture = Fixture::new(); + fixture.write_source("file.txt", b"old"); + fixture.write_dump("file.txt", b"new"); + let baseline = fixture.source_manifest(&["file.txt"]); + let plan = plan_manifest_merge(&baseline, &baseline, &fixture.dump_manifest(&["file.txt"])); + fixture + .request(&plan) + .prepare() + .unwrap() + .apply() + .unwrap() + .leave_for_recovery(); + fs::write(fixture.source.join("file.txt"), b"foreign").unwrap(); + + let error = recover_publication( + &fixture.transaction, + &fixture.source, + &TargetIdentity::new("ib-a"), + fixture.observed(6), + ) + .expect_err("foreign"); + + assert!(matches!(error, PublicationError::ForeignContent { .. })); + assert_eq!( + fs::read(fixture.source.join("file.txt")).unwrap(), + b"foreign" + ); + assert!(fixture.transaction.exists()); + } + + #[test] + fn visible_generation_recovery_preserves_later_user_edit() { + let fixture = Fixture::new(); + fixture.write_source("file.txt", b"old"); + fixture.write_dump("file.txt", b"new"); + let baseline = fixture.source_manifest(&["file.txt"]); + let plan = plan_manifest_merge(&baseline, &baseline, &fixture.dump_manifest(&["file.txt"])); + fixture + .request(&plan) + .prepare() + .unwrap() + .apply() + .unwrap() + .leave_for_recovery(); + fs::write(fixture.source.join("file.txt"), b"later-user-edit").unwrap(); + + recover_publication( + &fixture.transaction, + &fixture.source, + &TargetIdentity::new("ib-a"), + fixture.observed(7), + ) + .expect("visible generation proves publication committed"); + + assert_eq!( + fs::read(fixture.source.join("file.txt")).unwrap(), + b"later-user-edit" + ); + assert!(!fixture.transaction.exists()); + } + + #[test] + fn same_generation_with_unrelated_transaction_rolls_back_source() { + let fixture = Fixture::new(); + fixture.write_source("file.txt", b"old"); + fixture.write_dump("file.txt", b"new"); + let baseline = fixture.source_manifest(&["file.txt"]); + let plan = plan_manifest_merge(&baseline, &baseline, &fixture.dump_manifest(&["file.txt"])); + fixture + .request(&plan) + .prepare() + .unwrap() + .apply() + .unwrap() + .leave_for_recovery(); + + recover_publication( + &fixture.transaction, + &fixture.source, + &TargetIdentity::new("ib-a"), + ObservedStateGeneration::with_dump_transaction(7, DumpTransactionId::new()), + ) + .expect("unrelated state must not prove this source publication"); + + assert_eq!(fs::read(fixture.source.join("file.txt")).unwrap(), b"old"); + assert!(!fixture.transaction.exists()); + } + + #[test] + fn publication_journal_rejects_non_uuid_transaction_id() { + let fixture = Fixture::new(); + fs::create_dir(&fixture.transaction).unwrap(); + fs::write( + fixture.transaction.join(super::JOURNAL_FILE), + serde_json::to_vec(&serde_json::json!({ + "generation": 7, + "target_identity": "ib-a", + "source_root": fs::canonicalize(&fixture.source).unwrap(), + "status": "prepared", + "actions": [], + "dump_transaction_id": "../not-a-uuid" + })) + .unwrap(), + ) + .unwrap(); + + recover_publication( + &fixture.transaction, + &fixture.source, + &TargetIdentity::new("ib-a"), + fixture.observed(6), + ) + .expect_err("transaction id must be validated before recovery"); + + assert!(fixture.transaction.exists()); + } + + #[cfg(unix)] + #[test] + fn parent_swap_after_safe_open_never_mutates_the_symlink_target() { + use std::os::unix::fs::symlink; + + let fixture = Fixture::new(); + fixture.write_source("managed/file.txt", b"old"); + fixture.write_dump("managed/file.txt", b"new"); + let outside = fixture._temp.path().join("outside"); + fs::create_dir(&outside).unwrap(); + fs::write(outside.join("file.txt"), b"old").unwrap(); + let baseline = fixture.source_manifest(&["managed/file.txt"]); + let plan = plan_manifest_merge( + &baseline, + &baseline, + &fixture.dump_manifest(&["managed/file.txt"]), + ); + let prepared = fixture.request(&plan).prepare().unwrap(); + let managed = fixture.source.join("managed"); + let displaced = fixture.source.join("managed-displaced"); + let outside_for_hook = outside.clone(); + AFTER_SAFE_PARENT_OPEN.with(|hook| { + *hook.borrow_mut() = Some(Box::new(move || { + fs::rename(&managed, &displaced).unwrap(); + symlink(&outside_for_hook, &managed).unwrap(); + })); + }); + + prepared.apply().expect_err("swapped managed parent"); + + assert_eq!(fs::read(outside.join("file.txt")).unwrap(), b"old"); + } + + #[cfg(unix)] + #[test] + fn publication_accepts_a_symlinked_configured_root() { + use std::os::unix::fs::symlink; + + let dir = tempfile::tempdir().unwrap(); + let real_source = dir.path().join("real-source"); + let configured_source = dir.path().join("configured-source"); + let dump = dir.path().join("dump"); + fs::create_dir(&real_source).unwrap(); + fs::create_dir(&dump).unwrap(); + fs::write(real_source.join("file.txt"), b"old").unwrap(); + fs::write(dump.join("file.txt"), b"new").unwrap(); + symlink(&real_source, &configured_source).unwrap(); + let baseline = manifest(&[("file.txt", b"old")]); + let source = baseline.clone(); + let dump_manifest = manifest(&[("file.txt", b"new")]); + let plan = plan_manifest_merge(&baseline, &source, &dump_manifest); + + PublicationRequest::builder(&configured_source, &dump, &plan) + .transaction_root(&dir.path().join("transaction")) + .generation(7) + .target_identity(TargetIdentity::new("ib-a")) + .dump_transaction_id(DumpTransactionId::new()) + .prepare() + .unwrap() + .apply() + .unwrap(); + + assert_eq!(fs::read(real_source.join("file.txt")).unwrap(), b"new"); + } + + #[test] + fn source_applied_journal_failure_rolls_back_live_source() { + let fixture = Fixture::new(); + fixture.write_source("file.txt", b"old"); + fixture.write_dump("file.txt", b"new"); + let baseline = fixture.source_manifest(&["file.txt"]); + let plan = plan_manifest_merge(&baseline, &baseline, &fixture.dump_manifest(&["file.txt"])); + FAIL_JOURNAL_STATUS.with(|slot| slot.set(Some(super::JournalStatus::SourceApplied))); + + fixture + .request(&plan) + .prepare() + .unwrap() + .apply() + .expect_err("journal write must fail"); + + assert_eq!(fs::read(fixture.source.join("file.txt")).unwrap(), b"old"); + } + + #[test] + fn apply_reports_both_publication_and_rollback_failures() { + let fixture = Fixture::new(); + fixture.write_source("file.txt", b"old"); + fixture.write_dump("file.txt", b"new"); + let baseline = fixture.source_manifest(&["file.txt"]); + let plan = plan_manifest_merge(&baseline, &baseline, &fixture.dump_manifest(&["file.txt"])); + let source = fixture.source.join("file.txt"); + BEFORE_ACTION_INSTALL.with(|hook| { + *hook.borrow_mut() = Some(Box::new(move || { + fs::write(source, b"foreign").unwrap(); + })); + }); + + let error = fixture + .request(&plan) + .prepare() + .unwrap() + .apply() + .expect_err("publication and rollback must both fail"); + + assert!(matches!( + error, + PublicationError::PublicationAndRollback { .. } + )); + assert_eq!( + fs::read(fixture.source.join("file.txt")).unwrap(), + b"foreign" + ); + } + + #[test] + fn rollback_preserves_foreign_empty_directory_created_after_prepare() { + let fixture = Fixture::new(); + fixture.write_dump("new/file.txt", b"new"); + let plan = plan_manifest_merge( + &BTreeMap::new(), + &BTreeMap::new(), + &fixture.dump_manifest(&["new/file.txt"]), + ); + fixture + .request(&plan) + .prepare() + .unwrap() + .leave_for_recovery(); + let foreign = fixture.source.join("new"); + fs::create_dir(&foreign).unwrap(); + + recover_publication( + &fixture.transaction, + &fixture.source, + &TargetIdentity::new("ib-a"), + fixture.observed(6), + ) + .expect("prepared publication rollback"); + + assert!(foreign.is_dir()); + } + + #[test] + fn recovery_rejects_ignored_action_path() { + let fixture = Fixture::new(); + fixture.write_source(".git/config", b"keep"); + fs::create_dir(&fixture.transaction).unwrap(); + let before = super::hash_hex(b"keep"); + let journal = serde_json::json!({ + "generation": 7, + "target_identity": "ib-a", + "dump_transaction_id": fixture.dump_transaction_id.clone(), + "source_root": fs::canonicalize(&fixture.source).unwrap(), + "status": "prepared", + "actions": [{ + "path": ".git/config", + "before": {"sha256": before}, + "after": {"sha256": null}, + "backup": "00000000.bin", + "payload": null, + "claim": ".v8-runner-publication-claim-owned" + }], + }); + fs::write( + fixture.transaction.join(super::JOURNAL_FILE), + serde_json::to_vec(&journal).unwrap(), + ) + .unwrap(); + + recover_publication( + &fixture.transaction, + &fixture.source, + &TargetIdentity::new("ib-a"), + fixture.observed(6), + ) + .expect_err("ignored path must be rejected"); + + assert_eq!( + fs::read(fixture.source.join(".git/config")).unwrap(), + b"keep" + ); + } + + struct Fixture { + _temp: tempfile::TempDir, + source: std::path::PathBuf, + dump: std::path::PathBuf, + transaction: std::path::PathBuf, + dump_transaction_id: DumpTransactionId, + } + + impl Fixture { + fn new() -> Self { + let temp = tempfile::tempdir().unwrap(); + let source = temp.path().join("source"); + let dump = temp.path().join("dump"); + fs::create_dir_all(&source).unwrap(); + fs::create_dir_all(&dump).unwrap(); + Self { + transaction: temp.path().join("transaction"), + _temp: temp, + source, + dump, + dump_transaction_id: DumpTransactionId::new(), + } + } + + fn write_source(&self, path: &str, bytes: &[u8]) { + write(&self.source, path, bytes); + } + fn write_dump(&self, path: &str, bytes: &[u8]) { + write(&self.dump, path, bytes); + } + fn source_manifest(&self, paths: &[&str]) -> BTreeMap { + disk_manifest(&self.source, paths) + } + fn dump_manifest(&self, paths: &[&str]) -> BTreeMap { + disk_manifest(&self.dump, paths) + } + fn request<'a>( + &'a self, + plan: &'a crate::use_cases::shadow_merge::ManifestMergePlan, + ) -> super::PublicationRequestBuilder<'a> { + PublicationRequest::builder(&self.source, &self.dump, plan) + .transaction_root(&self.transaction) + .generation(7) + .target_identity(TargetIdentity::new("ib-a")) + .dump_transaction_id(self.dump_transaction_id.clone()) + } + + fn observed(&self, generation: u64) -> ObservedStateGeneration { + ObservedStateGeneration::with_dump_transaction( + generation, + self.dump_transaction_id.clone(), + ) + } + } + + fn write(root: &std::path::Path, path: &str, bytes: &[u8]) { + let target = root.join(path); + fs::create_dir_all(target.parent().unwrap()).unwrap(); + fs::write(target, bytes).unwrap(); + } + + fn disk_manifest(root: &std::path::Path, paths: &[&str]) -> BTreeMap { + paths + .iter() + .map(|path| { + ( + (*path).to_owned(), + hash(&fs::read(root.join(path)).unwrap()), + ) + }) + .collect() + } + + fn manifest(entries: &[(&str, &[u8])]) -> BTreeMap { + entries + .iter() + .map(|(path, bytes)| ((*path).to_owned(), hash(bytes))) + .collect() + } + + fn hash(bytes: &[u8]) -> [u8; 32] { + Sha256::digest(bytes).into() + } +} diff --git a/src/use_cases/source_transaction.rs b/src/use_cases/source_transaction.rs new file mode 100644 index 0000000..f5a8e3a --- /dev/null +++ b/src/use_cases/source_transaction.rs @@ -0,0 +1,817 @@ +use std::collections::{HashMap, HashSet}; +use std::fs::{self, OpenOptions}; +use std::io; +use std::path::{Path, PathBuf}; + +use tempfile::TempDir; +use thiserror::Error; +use walkdir::WalkDir; + +use crate::change_detection::analyzer::PreparedStateUpdate; +use crate::change_detection::scanner::{portable_relative_path, ScanError, SourceInventoryPolicy}; +use crate::use_cases::runtime_state::ValidatedCdfi; + +#[derive(Debug)] +pub(crate) enum CdfiSeed<'a> { + None, + Validated(&'a ValidatedCdfi), +} + +#[derive(Debug, Error)] +pub(crate) enum SourceTransactionError { + #[error("failed to prepare Designer source transaction: {0}")] + Io(#[from] std::io::Error), + #[error("failed to {operation} '{path}': {source}")] + FileIo { + operation: &'static str, + path: PathBuf, + source: std::io::Error, + }, + #[error("failed to inspect Designer source transaction: {0}")] + Walk(#[from] walkdir::Error), + #[error("invalid Designer source inventory: {0}")] + Inventory(#[from] ScanError), + #[error("staged Designer source differs from the planned source snapshot: {0}")] + SnapshotMismatch(String), +} + +pub(crate) struct DesignerSourceTransaction { + _transaction: TempDir, + load_root: PathBuf, +} + +impl DesignerSourceTransaction { + pub(crate) fn create( + source_root: &Path, + excluded_roots: &[PathBuf], + transactions_dir: &Path, + cdfi_seed: CdfiSeed<'_>, + ) -> Result { + let transactions_existed = transactions_dir.try_exists()?; + fs::create_dir_all(transactions_dir)?; + if !transactions_existed { + if let Some(parent) = transactions_dir.parent() { + sync_directory(parent)?; + } + } + let transaction = tempfile::Builder::new() + .prefix("designer-build-") + .tempdir_in(transactions_dir)?; + let load_root = transaction.path().join("source"); + fs::create_dir(&load_root)?; + sync_directory(transactions_dir)?; + let canonical_source_root = fs::canonicalize(source_root) + .map_err(|error| file_io("canonicalize source root", source_root, error))?; + let canonical_excluded_roots = excluded_roots + .iter() + .map(|excluded| { + excluded + .strip_prefix(source_root) + .map(|relative| canonical_source_root.join(relative)) + .unwrap_or_else(|_| excluded.clone()) + }) + .collect::>(); + let policy = SourceInventoryPolicy::new(&canonical_source_root, &canonical_excluded_roots)?; + for entry in WalkDir::new(&canonical_source_root) + .follow_links(false) + .into_iter() + .filter_entry(|entry| { + !entry.file_type().is_dir() || policy.should_descend(entry.path()) + }) + { + let entry = entry?; + if !entry.file_type().is_file() || !policy.includes_file(entry.path()) { + continue; + } + let relative = portable_relative_path(&canonical_source_root, entry.path())?; + let target = load_root.join(&relative); + if let Some(parent) = target.parent() { + fs::create_dir_all(parent)?; + } + copy_regular_no_follow( + &canonical_source_root, + &canonical_source_root, + Path::new(&relative), + entry.path(), + &target, + )?; + } + if let CdfiSeed::Validated(cdfi) = cdfi_seed { + fs::write(load_root.join("ConfigDumpInfo.xml"), cdfi.bytes())?; + } + Ok(Self { + _transaction: transaction, + load_root, + }) + } + + pub(crate) fn load_root(&self) -> &Path { + &self.load_root + } + + #[cfg(test)] + pub(crate) fn transaction_root(&self) -> &Path { + self._transaction.path() + } + + pub(crate) fn verify_snapshot( + &self, + prepared: &PreparedStateUpdate, + ) -> Result<(), SourceTransactionError> { + if let Some((planned, actual)) = snapshot_mismatch_counts(&self.load_root, &[], prepared)? { + Err(SourceTransactionError::SnapshotMismatch(format!( + "planned {} file(s), staged {} file(s)", + planned, actual + ))) + } else { + Ok(()) + } + } + + pub(crate) fn close(self) -> Result<(), SourceTransactionError> { + let parent = self._transaction.path().parent().map(Path::to_path_buf); + self._transaction.close()?; + if let Some(parent) = parent { + sync_directory(&parent)?; + } + Ok(()) + } +} + +pub(crate) fn verify_source_snapshot( + source_root: &Path, + excluded_roots: &[PathBuf], + prepared: &PreparedStateUpdate, +) -> Result<(), SourceTransactionError> { + if let Some((planned, actual)) = + snapshot_mismatch_counts(source_root, excluded_roots, prepared)? + { + Err(SourceTransactionError::SnapshotMismatch(format!( + "planned {} file(s), observed {} file(s) in '{}'", + planned, + actual, + source_root.display() + ))) + } else { + Ok(()) + } +} + +fn snapshot_mismatch_counts( + source_root: &Path, + excluded_roots: &[PathBuf], + prepared: &PreparedStateUpdate, +) -> Result, SourceTransactionError> { + let scan = + crate::change_detection::scanner::scan(source_root, None, &HashSet::new(), excluded_roots)?; + let actual = scan + .candidates + .iter() + .map(|file| (file.rel_path.as_str(), file.hash.as_str())) + .collect::>(); + let planned = prepared + .snapshot + .iter() + .map(|file| (file.rel_path.as_str(), file.hash.as_str())) + .collect::>(); + Ok((actual != planned).then_some((planned.len(), actual.len()))) +} + +fn copy_regular_no_follow( + source_root: &Path, + _canonical_source_root: &Path, + relative: &Path, + source: &Path, + target: &Path, +) -> Result<(), SourceTransactionError> { + #[cfg(unix)] + return copy_regular_beneath_unix(source_root, relative, source, target, || {}); + #[cfg(not(unix))] + copy_regular_no_follow_portable(_canonical_source_root, source, target, || {}) +} + +#[cfg(test)] +fn copy_regular_no_follow_with_hook( + canonical_source_root: &Path, + source: &Path, + target: &Path, + before_open: F, +) -> Result<(), SourceTransactionError> +where + F: FnOnce(), +{ + #[cfg(unix)] + { + let relative = source.strip_prefix(canonical_source_root).map_err(|_| { + io::Error::new( + io::ErrorKind::InvalidInput, + format!( + "source '{}' is outside managed root '{}'", + source.display(), + canonical_source_root.display() + ), + ) + })?; + return copy_regular_beneath_unix( + canonical_source_root, + relative, + source, + target, + before_open, + ); + } + #[cfg(not(unix))] + copy_regular_no_follow_portable(canonical_source_root, source, target, before_open) +} + +#[cfg(not(unix))] +fn copy_regular_no_follow_portable( + canonical_source_root: &Path, + source: &Path, + target: &Path, + before_open: F, +) -> Result<(), SourceTransactionError> +where + F: FnOnce(), +{ + validate_source_containment(canonical_source_root, source)?; + let before = fs::symlink_metadata(source) + .map_err(|error| file_io("inspect source file", source, error))?; + if !before.file_type().is_file() + || before.file_type().is_symlink() + || metadata_is_reparse_point(&before) + { + return Err(io::Error::new( + io::ErrorKind::InvalidData, + format!("source '{}' is not a regular file", source.display()), + ) + .into()); + } + #[cfg(windows)] + let before_handle_identity = windows_path_identity(source)?; + before_open(); + let mut options = OpenOptions::new(); + options.read(true); + #[cfg(windows)] + { + use std::os::windows::fs::OpenOptionsExt; + const FILE_FLAG_OPEN_REPARSE_POINT: u32 = 0x0020_0000; + options.custom_flags(FILE_FLAG_OPEN_REPARSE_POINT); + } + let mut input = options + .open(source) + .map_err(|error| file_io("open source file without following symlinks", source, error))?; + let opened = input + .metadata() + .map_err(|error| file_io("inspect opened source file", source, error))?; + #[cfg(windows)] + let windows_identity_changed = windows_file_identity(&input, source)? != before_handle_identity; + #[cfg(not(windows))] + let windows_identity_changed = false; + if !opened.file_type().is_file() + || opened.file_type().is_symlink() + || metadata_is_reparse_point(&opened) + || !same_file_identity(&before, &opened) + || windows_identity_changed + { + return Err(io::Error::new( + io::ErrorKind::InvalidData, + format!("source '{}' changed while staging", source.display()), + ) + .into()); + } + let mut output = OpenOptions::new() + .write(true) + .create_new(true) + .open(target) + .map_err(|error| file_io("create staged file", target, error))?; + io::copy(&mut input, &mut output) + .map_err(|error| file_io("copy source into stage", target, error))?; + output + .sync_all() + .map_err(|error| file_io("sync staged file", target, error))?; + let after = input + .metadata() + .map_err(|error| file_io("re-inspect opened source file", source, error))?; + validate_source_containment(canonical_source_root, source)?; + if !same_file_identity(&opened, &after) || metadata_is_reparse_point(&after) { + return Err(io::Error::new( + io::ErrorKind::InvalidData, + format!("source '{}' changed during staging", source.display()), + ) + .into()); + } + Ok(()) +} + +#[cfg(unix)] +fn copy_regular_beneath_unix( + source_root: &Path, + relative: &Path, + source: &Path, + target: &Path, + before_open: F, +) -> Result<(), SourceTransactionError> +where + F: FnOnce(), +{ + use std::ffi::CString; + use std::os::fd::{AsRawFd, FromRawFd}; + use std::os::unix::ffi::OsStrExt; + use std::os::unix::fs::OpenOptionsExt; + + let mut root_options = OpenOptions::new(); + root_options + .read(true) + .custom_flags(libc::O_DIRECTORY | libc::O_NOFOLLOW | libc::O_CLOEXEC); + let mut directory = root_options.open(source_root).map_err(|error| { + file_io( + "open source root without following symlinks", + source_root, + error, + ) + })?; + before_open(); + + let components = relative.components().collect::>(); + let Some((file_component, parent_components)) = components.split_last() else { + return Err(io::Error::new(io::ErrorKind::InvalidInput, "empty source path").into()); + }; + for component in parent_components { + let std::path::Component::Normal(name) = component else { + return Err(io::Error::new( + io::ErrorKind::InvalidInput, + format!( + "source '{}' has a non-normal path component", + source.display() + ), + ) + .into()); + }; + let name = CString::new(name.as_bytes()).map_err(|_| { + io::Error::new( + io::ErrorKind::InvalidInput, + "source path contains a NUL byte", + ) + })?; + // SAFETY: `directory` is a live directory descriptor and `name` is a + // NUL-terminated single component. O_NOFOLLOW rejects a swapped symlink. + let descriptor = unsafe { + libc::openat( + directory.as_raw_fd(), + name.as_ptr(), + libc::O_RDONLY | libc::O_DIRECTORY | libc::O_NOFOLLOW | libc::O_CLOEXEC, + ) + }; + if descriptor < 0 { + return Err(file_io( + "open source directory component without following symlinks", + source, + io::Error::last_os_error(), + )); + } + // SAFETY: `openat` returned a new owned descriptor. + directory = unsafe { fs::File::from_raw_fd(descriptor) }; + } + + let std::path::Component::Normal(file_name) = file_component else { + return Err(io::Error::new( + io::ErrorKind::InvalidInput, + format!( + "source '{}' has a non-normal file component", + source.display() + ), + ) + .into()); + }; + let file_name = CString::new(file_name.as_bytes()).map_err(|_| { + io::Error::new( + io::ErrorKind::InvalidInput, + "source path contains a NUL byte", + ) + })?; + // SAFETY: same descriptor/name invariants as the directory traversal above. + let descriptor = unsafe { + libc::openat( + directory.as_raw_fd(), + file_name.as_ptr(), + libc::O_RDONLY | libc::O_NOFOLLOW | libc::O_CLOEXEC, + ) + }; + if descriptor < 0 { + return Err(file_io( + "open source file beneath root without following symlinks", + source, + io::Error::last_os_error(), + )); + } + // SAFETY: `openat` returned a new owned descriptor. + let mut input = unsafe { fs::File::from_raw_fd(descriptor) }; + let opened = input + .metadata() + .map_err(|error| file_io("inspect opened source file", source, error))?; + if !opened.file_type().is_file() { + return Err(io::Error::new( + io::ErrorKind::InvalidData, + format!("source '{}' is not a regular file", source.display()), + ) + .into()); + } + + let mut output = OpenOptions::new() + .write(true) + .create_new(true) + .open(target) + .map_err(|error| file_io("create staged file", target, error))?; + io::copy(&mut input, &mut output) + .map_err(|error| file_io("copy source into stage", target, error))?; + output + .sync_all() + .map_err(|error| file_io("sync staged file", target, error))?; + let after = input + .metadata() + .map_err(|error| file_io("re-inspect opened source file", source, error))?; + if !same_file_identity(&opened, &after) { + return Err(io::Error::new( + io::ErrorKind::InvalidData, + format!("source '{}' changed during staging", source.display()), + ) + .into()); + } + Ok(()) +} + +#[cfg(not(unix))] +fn validate_source_containment( + canonical_source_root: &Path, + source: &Path, +) -> Result<(), SourceTransactionError> { + let canonical_source = fs::canonicalize(source) + .map_err(|error| file_io("canonicalize source file", source, error))?; + if canonical_source.starts_with(canonical_source_root) { + Ok(()) + } else { + Err(io::Error::new( + io::ErrorKind::InvalidData, + format!( + "source '{}' resolves outside managed root '{}'", + source.display(), + canonical_source_root.display() + ), + ) + .into()) + } +} + +fn file_io(operation: &'static str, path: &Path, source: io::Error) -> SourceTransactionError { + SourceTransactionError::FileIo { + operation, + path: path.to_path_buf(), + source, + } +} + +#[cfg(unix)] +fn same_file_identity(left: &fs::Metadata, right: &fs::Metadata) -> bool { + use std::os::unix::fs::MetadataExt; + left.dev() == right.dev() && left.ino() == right.ino() && left.len() == right.len() +} + +#[cfg(windows)] +fn same_file_identity(left: &fs::Metadata, right: &fs::Metadata) -> bool { + // Stable identity is checked with GetFileInformationByHandle; metadata still + // detects a size change during the copy. + left.len() == right.len() +} + +#[cfg(not(any(unix, windows)))] +fn same_file_identity(left: &fs::Metadata, right: &fs::Metadata) -> bool { + left.len() == right.len() && left.modified().ok() == right.modified().ok() +} + +#[cfg(windows)] +fn metadata_is_reparse_point(metadata: &fs::Metadata) -> bool { + use std::os::windows::fs::MetadataExt; + const FILE_ATTRIBUTE_REPARSE_POINT: u32 = 0x0000_0400; + metadata.file_attributes() & FILE_ATTRIBUTE_REPARSE_POINT != 0 +} + +#[cfg(windows)] +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +struct WindowsFileIdentity { + volume_serial: u32, + file_index: u64, +} + +#[cfg(windows)] +fn windows_path_identity(path: &Path) -> Result { + use std::os::windows::fs::OpenOptionsExt; + const FILE_FLAG_OPEN_REPARSE_POINT: u32 = 0x0020_0000; + let file = OpenOptions::new() + .read(true) + .custom_flags(FILE_FLAG_OPEN_REPARSE_POINT) + .open(path) + .map_err(|error| file_io("open source identity handle", path, error))?; + let metadata = file + .metadata() + .map_err(|error| file_io("inspect source identity handle", path, error))?; + if metadata_is_reparse_point(&metadata) || !metadata.file_type().is_file() { + return Err(io::Error::new( + io::ErrorKind::InvalidData, + format!("source '{}' is a Windows reparse point", path.display()), + ) + .into()); + } + windows_file_identity(&file, path) +} + +#[cfg(windows)] +fn windows_file_identity( + file: &fs::File, + path: &Path, +) -> Result { + use std::mem::MaybeUninit; + use std::os::windows::io::AsRawHandle; + use windows_sys::Win32::Storage::FileSystem::{ + GetFileInformationByHandle, BY_HANDLE_FILE_INFORMATION, + }; + + let mut information = MaybeUninit::::uninit(); + // SAFETY: the handle is borrowed from a live File, and Windows initializes + // BY_HANDLE_FILE_INFORMATION when the call succeeds. + let succeeded = unsafe { + GetFileInformationByHandle(file.as_raw_handle().cast(), information.as_mut_ptr()) + }; + if succeeded == 0 { + return Err(file_io( + "query Windows source file identity", + path, + io::Error::last_os_error(), + )); + } + // SAFETY: guarded by the successful Win32 call above. + let information = unsafe { information.assume_init() }; + Ok(WindowsFileIdentity { + volume_serial: information.dwVolumeSerialNumber, + file_index: (u64::from(information.nFileIndexHigh) << 32) + | u64::from(information.nFileIndexLow), + }) +} + +#[cfg(not(any(unix, windows)))] +fn metadata_is_reparse_point(_metadata: &fs::Metadata) -> bool { + false +} + +#[cfg(unix)] +fn sync_directory(path: &Path) -> Result<(), SourceTransactionError> { + fs::File::open(path)?.sync_all()?; + Ok(()) +} + +#[cfg(not(unix))] +fn sync_directory(_path: &Path) -> Result<(), SourceTransactionError> { + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::{copy_regular_no_follow_with_hook, CdfiSeed, DesignerSourceTransaction}; + use crate::change_detection::analyzer::{PreparedFileState, PreparedStateUpdate}; + use crate::change_detection::hash_storage::ObservedStorageState; + use std::fs; + use tempfile::tempdir; + + #[test] + fn copies_only_regular_managed_files_and_cleans_stage_on_drop() { + let dir = tempdir().expect("tempdir"); + let source = dir.path().join("source"); + let work = source.join("nested-work"); + let transactions = dir.path().join("state/transactions"); + fs::create_dir_all(source.join("Catalogs")).expect("source dirs"); + fs::create_dir_all(&work).expect("nested work"); + fs::write(source.join("Configuration.xml"), b"config").expect("config"); + fs::write(source.join("Catalogs/Item.xml"), b"item").expect("item"); + fs::write(source.join("CONFIGDUMPINFO.XML"), b"user-owned").expect("cdfi"); + fs::write(work.join("secret.txt"), b"state").expect("work file"); + + let stage_path = { + let transaction = + DesignerSourceTransaction::create(&source, &[work], &transactions, CdfiSeed::None) + .expect("transaction"); + assert_eq!( + fs::read(transaction.load_root().join("Configuration.xml")).expect("staged"), + b"config" + ); + assert_eq!( + fs::read(transaction.load_root().join("Catalogs/Item.xml")).expect("staged"), + b"item" + ); + assert!(!transaction.load_root().join("CONFIGDUMPINFO.XML").exists()); + assert!(!transaction.load_root().join("nested-work").exists()); + transaction.transaction_root().to_path_buf() + }; + + assert!(!stage_path.exists()); + } + + #[test] + fn seeds_only_prevalidated_private_cdfi() { + let dir = tempdir().expect("tempdir"); + let source = dir.path().join("source"); + let private = dir.path().join("private.xml"); + fs::create_dir_all(&source).expect("source"); + fs::write(source.join("Configuration.xml"), b"config").expect("config"); + fs::write( + &private, + r#""#, + ) + .expect("private CDFI"); + let validated = match crate::use_cases::runtime_state::inspect_private_cdfi(&private) + .expect("inspect") + { + crate::use_cases::runtime_state::PrivateCdfiState::Valid(cdfi) => cdfi, + state => panic!("expected valid CDFI, got {state:?}"), + }; + + let transaction = DesignerSourceTransaction::create( + &source, + &[], + &dir.path().join("transactions"), + CdfiSeed::Validated(&validated), + ) + .expect("transaction"); + assert_eq!( + fs::read(transaction.load_root().join("ConfigDumpInfo.xml")).expect("seed"), + fs::read(private).expect("private") + ); + } + + #[test] + fn rejects_stage_that_differs_from_planned_snapshot() { + let dir = tempdir().expect("tempdir"); + let source = dir.path().join("source"); + fs::create_dir_all(&source).expect("source"); + fs::write(source.join("Configuration.xml"), b"actual").expect("source file"); + let transaction = DesignerSourceTransaction::create( + &source, + &[], + &dir.path().join("transactions"), + CdfiSeed::None, + ) + .expect("transaction"); + let planned = PreparedStateUpdate { + snapshot: vec![PreparedFileState { + rel_path: "Configuration.xml".to_owned(), + mtime_ns: 0, + hash: "0".repeat(64), + }], + scan_started_at: 0, + observed_storage: ObservedStorageState::MissingPath, + }; + + assert!(transaction.verify_snapshot(&planned).is_err()); + } + + #[cfg(unix)] + #[test] + fn does_not_follow_file_or_directory_symlinks() { + use std::os::unix::fs::symlink; + + let dir = tempdir().expect("tempdir"); + let source = dir.path().join("source"); + let outside = dir.path().join("outside"); + fs::create_dir_all(&source).expect("source"); + fs::create_dir_all(&outside).expect("outside"); + fs::write(outside.join("file.xml"), b"outside").expect("outside file"); + symlink(outside.join("file.xml"), source.join("file-link.xml")).expect("file symlink"); + symlink(&outside, source.join("dir-link")).expect("dir symlink"); + + let transaction = DesignerSourceTransaction::create( + &source, + &[], + &dir.path().join("transactions"), + CdfiSeed::None, + ) + .expect("transaction"); + assert!(!transaction.load_root().join("file-link.xml").exists()); + assert!(!transaction.load_root().join("dir-link").exists()); + } + + #[cfg(unix)] + #[test] + fn accepts_symlinked_source_root_without_following_nested_symlinks() { + use std::os::unix::fs::symlink; + + let dir = tempdir().expect("tempdir"); + let physical_source = dir.path().join("physical-source"); + let source_link = dir.path().join("source-link"); + let outside = dir.path().join("outside.xml"); + let excluded = physical_source.join("excluded"); + fs::create_dir_all(&physical_source).expect("physical source"); + fs::create_dir_all(&excluded).expect("excluded source"); + fs::write(physical_source.join("Configuration.xml"), b"config").expect("source file"); + fs::write(excluded.join("secret.xml"), b"secret").expect("excluded file"); + fs::write(&outside, b"outside").expect("outside file"); + symlink(&outside, physical_source.join("outside-link.xml")).expect("nested symlink"); + symlink(&physical_source, &source_link).expect("source root symlink"); + + let transaction = DesignerSourceTransaction::create( + &source_link, + &[source_link.join("excluded")], + &dir.path().join("transactions"), + CdfiSeed::None, + ) + .expect("symlinked source root"); + + assert_eq!( + fs::read(transaction.load_root().join("Configuration.xml")).expect("staged source"), + b"config" + ); + assert!(!transaction.load_root().join("excluded").exists()); + assert!(!transaction.load_root().join("outside-link.xml").exists()); + } + + #[cfg(unix)] + #[test] + fn rejects_regular_file_replaced_by_symlink_before_open() { + use std::os::unix::fs::symlink; + + let dir = tempdir().expect("tempdir"); + let source = dir.path().join("source.xml"); + let outside = dir.path().join("outside.xml"); + let target = dir.path().join("target.xml"); + fs::write(&source, b"inside").expect("source"); + fs::write(&outside, b"outside-secret").expect("outside"); + + let canonical_root = fs::canonicalize(dir.path()).expect("canonical root"); + let error = copy_regular_no_follow_with_hook(&canonical_root, &source, &target, || { + fs::remove_file(&source).expect("remove source"); + symlink(&outside, &source).expect("replace by symlink"); + }) + .expect_err("symlink swap must fail"); + + assert!(error.to_string().contains("source")); + assert!(!target.exists()); + } + + #[cfg(unix)] + #[test] + fn rejects_intermediate_directory_replaced_by_symlink() { + use std::os::unix::fs::symlink; + + let dir = tempdir().expect("tempdir"); + let root = dir.path().join("root"); + let nested = root.join("nested"); + let moved = root.join("nested-old"); + let attack_link = root.join("attack-link"); + let source = nested.join("source.xml"); + let target = dir.path().join("target.xml"); + fs::create_dir_all(&nested).expect("nested"); + fs::write(&source, b"inside").expect("inside"); + let canonical_root = fs::canonicalize(&root).expect("canonical root"); + + let result = copy_regular_no_follow_with_hook(&canonical_root, &source, &target, || { + fs::rename(&nested, &moved).expect("move nested"); + symlink(&moved, &nested).expect("replace intermediate directory"); + }); + fs::rename(&nested, &attack_link).expect("move attack symlink aside"); + if moved.try_exists().expect("inspect moved directory") { + fs::rename(&moved, &nested).expect("restore nested directory"); + } else { + fs::rename(&attack_link, &nested).expect("restore moved directory"); + } + + result.expect_err("intermediate symlink swap to the same file must fail"); + + assert!(!target.exists()); + } + + #[cfg(windows)] + #[test] + fn rejects_regular_file_replaced_by_windows_symlink_before_open() { + use std::os::windows::fs::symlink_file; + + let dir = tempdir().expect("tempdir"); + let source = dir.path().join("source.xml"); + let outside = dir.path().join("outside.xml"); + let target = dir.path().join("target.xml"); + fs::write(&source, b"inside").expect("source"); + fs::write(&outside, b"outside-secret").expect("outside"); + let probe = dir.path().join("probe-link.xml"); + if symlink_file(&outside, &probe).is_err() { + // Windows requires Developer Mode or SeCreateSymbolicLinkPrivilege. + return; + } + fs::remove_file(&probe).expect("remove probe"); + let canonical_root = fs::canonicalize(dir.path()).expect("canonical root"); + + copy_regular_no_follow_with_hook(&canonical_root, &source, &target, || { + fs::remove_file(&source).expect("remove source"); + symlink_file(&outside, &source).expect("replace by symlink"); + }) + .expect_err("Windows reparse swap must fail"); + + assert!(!target.exists()); + } +} diff --git a/src/use_cases/tool_extension.rs b/src/use_cases/tool_extension.rs index 2b1229c..118644a 100644 --- a/src/use_cases/tool_extension.rs +++ b/src/use_cases/tool_extension.rs @@ -9,7 +9,12 @@ use crate::config::model::{ ToolExtensionSourceConfig, }; use crate::domain::build::{BuildMode, BuildStep}; +use crate::domain::runtime_state::{ + InfobaseIdentity, LogicalSourceRole, RuntimeSourceDescriptor, RuntimeSourceIdentityInputs, + RuntimeStateLayout, +}; use crate::domain::source_set::SourceSetContext; +use crate::domain::sync_receipt::{SyncReceipt, SyncTarget}; use crate::platform::designer::DesignerDsl; use crate::platform::edt::EdtDsl; use crate::platform::edt_session::{EdtSessionHostOptions, EdtSessionManager}; @@ -25,6 +30,14 @@ use crate::use_cases::build_progress::{log_timeline_stage, TimelineStageStatus}; use crate::use_cases::context::{ExecutionContext, InterruptionSafetyClass}; use crate::use_cases::ibcmd_diagnostics::format_ibcmd_failure_details; use crate::use_cases::interruption; +use crate::use_cases::runtime_state::{ + cleanup_orphan_designer_transactions, commit_designer_state_with_lock, + designer_full_rebuild_required, inspect_private_cdfi, lock_designer_state, + recover_designer_state_with_lock, require_designer_full_rebuild, PrivateCdfiState, +}; +use crate::use_cases::source_transaction::{ + verify_source_snapshot, CdfiSeed, DesignerSourceTransaction, +}; #[derive(Debug)] pub(crate) struct ToolExtensionFailure { @@ -32,6 +45,25 @@ pub(crate) struct ToolExtensionFailure { pub(crate) error: AppError, } +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum ToolCdfiSeedPolicy { + Allow, + Omit, +} + +struct ToolSourceCommit<'a> { + context: &'a SourceSetContext, + prepared: &'a analyzer::PreparedStateUpdate, + state_commit: ToolStateCommit, + cdfi_seed: ToolCdfiSeedPolicy, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum ToolStateCommit { + Prepared, + FullObservation, +} + pub(crate) fn client_mcp_extension(config: &AppConfig) -> Option<&ToolExtensionConfig> { config.tools.client_mcp.extension.as_ref() } @@ -56,15 +88,7 @@ pub(crate) fn prepare_client_mcp_extension( let Some(extension) = client_mcp_extension(config) else { return Ok(None); }; - prepare_extension(context, config, extension, full_rebuild) - .map(Some) - .map_err(|error| { - let message = error.to_string(); - ToolExtensionFailure { - step: failed_build_step(extension, message), - error, - } - }) + prepare_extension(context, config, extension, full_rebuild).map(Some) } fn prepare_extension( @@ -72,11 +96,11 @@ fn prepare_extension( config: &AppConfig, extension: &ToolExtensionConfig, full_rebuild: bool, -) -> Result { +) -> Result { let started = Instant::now(); if let Some(error) = interruption_before_safe_point(context, extension, "tool extension load") { - return Err(error); + return Err(tool_extension_failure(extension, error, vec![])); } let mut utilities = PlatformUtilities::from_config(config); @@ -91,7 +115,8 @@ fn prepare_extension( full_rebuild, ), ToolExtensionInput::Artifact(artifact) => { - prepare_artifact_extension(context, config, extension, &artifact.path, &mut utilities)?; + prepare_artifact_extension(context, config, extension, &artifact.path, &mut utilities) + .map_err(|error| tool_extension_failure(extension, error, vec![]))?; Ok(successful_build_step( extension, format!("prepared extension '{}' from .cfe artifact", extension.name), @@ -109,30 +134,101 @@ fn prepare_source_extension( utilities: &mut PlatformUtilities, started: Instant, full_rebuild: bool, -) -> Result { - let source_context = tool_extension_source_context(config, extension, source)?; +) -> Result { + let source_context = tool_extension_source_context(config, extension, source) + .map_err(|error| tool_extension_failure(extension, error, vec![]))?; + let cdfi_requires_bootstrap = if config.builder == BuilderBackend::Designer { + let state_lock = lock_designer_state(&source_context).map_err(|error| { + tool_extension_failure(extension, AppError::Runtime(error.to_string()), vec![]) + })?; + recover_designer_state_with_lock(&source_context, &state_lock).map_err(|error| { + tool_extension_failure(extension, AppError::Runtime(error.to_string()), vec![]) + })?; + cleanup_orphan_designer_transactions(&source_context, &state_lock).map_err(|error| { + tool_extension_failure(extension, AppError::Runtime(error.to_string()), vec![]) + })?; + designer_full_rebuild_required(&source_context).map_err(|error| { + tool_extension_failure(extension, AppError::Runtime(error.to_string()), vec![]) + })? || !matches!( + inspect_private_cdfi(&source_context.private_cdfi_path()).map_err(|error| { + tool_extension_failure(extension, AppError::Runtime(error.to_string()), vec![]) + })?, + PrivateCdfiState::Valid(_) + ) + } else { + false + }; if full_rebuild { - prepare_source_extension_full(context, config, extension, source, utilities)?; - commit_tool_extension_full_rescan(&source_context, &config.work_path, true)?; - return Ok(successful_build_step( + let (requested, processed, prepared) = full_inventory_receipt(&source_context) + .map_err(|error| tool_extension_failure(extension, error, vec![]))?; + let receipt = applied_receipt(requested.clone(), processed) + .map_err(|error| tool_extension_failure(extension, error, requested.clone()))?; + let commit = ToolSourceCommit { + context: &source_context, + prepared: &prepared, + state_commit: ToolStateCommit::FullObservation, + cdfi_seed: ToolCdfiSeedPolicy::Omit, + }; + prepare_source_extension_full(context, config, extension, source, utilities, &commit) + .map_err(|error| tool_extension_failure(extension, error, requested.clone()))?; + return Ok(successful_build_step_with_receipt( + extension, + format!("prepared extension '{}' from sources", extension.name), + started.elapsed().as_millis() as u64, + receipt, + )); + } + + if cdfi_requires_bootstrap { + let (requested, processed, prepared) = full_inventory_receipt(&source_context) + .map_err(|error| tool_extension_failure(extension, error, vec![]))?; + let receipt = applied_receipt(requested.clone(), processed) + .map_err(|error| tool_extension_failure(extension, error, requested.clone()))?; + let commit = ToolSourceCommit { + context: &source_context, + prepared: &prepared, + state_commit: ToolStateCommit::FullObservation, + cdfi_seed: ToolCdfiSeedPolicy::Omit, + }; + prepare_source_extension_full(context, config, extension, source, utilities, &commit) + .map_err(|error| tool_extension_failure(extension, error, requested.clone()))?; + return Ok(successful_build_step_with_receipt( extension, format!("prepared extension '{}' from sources", extension.name), started.elapsed().as_millis() as u64, + receipt, )); } - let outcome = match analyzer::analyze_context(&source_context, &config.work_path).outcome { + let outcome = match analyzer::analyze_context(&source_context).outcome { Ok(outcome) => outcome, - Err(error) if storage_needs_recovery(&source_context, &config.work_path) => { - prepare_source_extension_full(context, config, extension, source, utilities)?; - commit_tool_extension_full_rescan(&source_context, &config.work_path, true)?; - return Ok(successful_build_step( + Err(_error) if storage_needs_recovery(&source_context) => { + let (requested, processed, prepared) = full_inventory_receipt(&source_context) + .map_err(|error| tool_extension_failure(extension, error, vec![]))?; + let receipt = applied_receipt(requested.clone(), processed) + .map_err(|error| tool_extension_failure(extension, error, requested.clone()))?; + let commit = ToolSourceCommit { + context: &source_context, + prepared: &prepared, + state_commit: ToolStateCommit::FullObservation, + cdfi_seed: ToolCdfiSeedPolicy::Allow, + }; + prepare_source_extension_full(context, config, extension, source, utilities, &commit) + .map_err(|error| tool_extension_failure(extension, error, requested.clone()))?; + return Ok(successful_build_step_with_receipt( extension, format!("prepared extension '{}' from sources", extension.name), started.elapsed().as_millis() as u64, + receipt, )); } - Err(error) => return Err(AppError::Runtime(error.to_string())), + Err(error) => { + return Err(tool_extension_failure( + extension, + AppError::Runtime(error.to_string()), + vec![], + )) + } }; match outcome { @@ -141,23 +237,44 @@ fn prepare_source_extension( "no changes".to_owned(), started.elapsed().as_millis() as u64, )), - AnalysisOutcome::Fallback => { - prepare_source_extension_full(context, config, extension, source, utilities)?; - commit_tool_extension_full_rescan(&source_context, &config.work_path, false)?; - Ok(successful_build_step( + AnalysisOutcome::Bootstrap | AnalysisOutcome::Fallback => { + let (requested, processed, prepared) = full_inventory_receipt(&source_context) + .map_err(|error| tool_extension_failure(extension, error, vec![]))?; + let receipt = applied_receipt(requested.clone(), processed) + .map_err(|error| tool_extension_failure(extension, error, requested.clone()))?; + let commit = ToolSourceCommit { + context: &source_context, + prepared: &prepared, + state_commit: ToolStateCommit::FullObservation, + cdfi_seed: ToolCdfiSeedPolicy::Allow, + }; + prepare_source_extension_full(context, config, extension, source, utilities, &commit) + .map_err(|error| tool_extension_failure(extension, error, requested.clone()))?; + Ok(successful_build_step_with_receipt( extension, format!("prepared extension '{}' from sources", extension.name), started.elapsed().as_millis() as u64, + receipt, )) } - AnalysisOutcome::Changes { prepared, .. } => { - prepare_source_extension_full(context, config, extension, source, utilities)?; - analyzer::commit_success(&source_context, &config.work_path, &prepared) - .map_err(|error| AppError::Runtime(error.to_string()))?; - Ok(successful_build_step( + AnalysisOutcome::Changes { changes, prepared } => { + let (requested, processed) = change_receipt(&changes, &prepared) + .map_err(|error| tool_extension_failure(extension, error, vec![]))?; + let receipt = applied_receipt(requested.clone(), processed) + .map_err(|error| tool_extension_failure(extension, error, requested.clone()))?; + let commit = ToolSourceCommit { + context: &source_context, + prepared: &prepared, + state_commit: ToolStateCommit::Prepared, + cdfi_seed: ToolCdfiSeedPolicy::Allow, + }; + prepare_source_extension_full(context, config, extension, source, utilities, &commit) + .map_err(|error| tool_extension_failure(extension, error, requested.clone()))?; + Ok(successful_build_step_with_receipt( extension, format!("prepared extension '{}' from sources", extension.name), started.elapsed().as_millis() as u64, + receipt, )) } } @@ -169,20 +286,63 @@ fn prepare_source_extension_full( extension: &ToolExtensionConfig, source: &ToolExtensionSourceConfig, utilities: &mut PlatformUtilities, + commit: &ToolSourceCommit<'_>, ) -> Result<(), AppError> { match source.format.unwrap_or(config.format) { - SourceFormat::Designer => { - prepare_designer_source_extension(context, config, extension, &source.path, utilities) - } + SourceFormat::Designer => prepare_source_extension_for_backend( + context, + config, + extension, + commit.context.path(), + utilities, + commit, + ), SourceFormat::Edt => { - let exported = - export_edt_source_extension(context, config, extension, &source.path, utilities)?; - prepare_designer_source_extension(context, config, extension, &exported, utilities) + let exported = export_edt_source_extension( + context, + config, + extension, + commit.context.path(), + utilities, + )?; + prepare_source_extension_for_backend( + context, config, extension, &exported, utilities, commit, + ) + } + } +} + +fn prepare_source_extension_for_backend( + context: &ExecutionContext, + config: &AppConfig, + extension: &ToolExtensionConfig, + source_path: &Path, + utilities: &mut PlatformUtilities, + commit: &ToolSourceCommit<'_>, +) -> Result<(), AppError> { + if config.builder == BuilderBackend::Designer && source_path != commit.context.path() { + verify_source_snapshot( + commit.context.path(), + commit.context.excluded_roots(), + commit.prepared, + ) + .map_err(|error| AppError::Runtime(error.to_string()))?; + } + prepare_designer_source_extension(context, config, extension, source_path, utilities, commit)?; + if config.builder == BuilderBackend::Ibcmd { + match commit.state_commit { + ToolStateCommit::FullObservation => { + commit_tool_extension_full_observation(commit.context, commit.prepared) + } + ToolStateCommit::Prepared => analyzer::commit_success(commit.context, commit.prepared) + .map_err(|error| AppError::Runtime(error.to_string())), } + } else { + Ok(()) } } -fn tool_extension_source_context( +pub(crate) fn tool_extension_source_context( config: &AppConfig, extension: &ToolExtensionConfig, source: &ToolExtensionSourceConfig, @@ -193,11 +353,21 @@ fn tool_extension_source_context( } else { base_path.join(&source.path) }; - Ok(SourceSetContext::new( - format!("tool:{}", extension.name), - source_path, - format!("tool-{}-source", extension.name), - )) + let identity = InfobaseIdentity::normalize(&config.infobase)?; + let layout = RuntimeStateLayout::new(&config.work_path, identity)?; + let descriptor = RuntimeSourceDescriptor::new(RuntimeSourceIdentityInputs { + configured_source_identity: &source.path, + source_root: &source_path, + purpose: crate::config::model::SourceSetPurpose::Extension, + format: source.format.unwrap_or(config.format), + backend: config.builder, + logical_role: LogicalSourceRole::ToolExtension, + })?; + let state = layout.source_state(&format!("tool-{}", extension.name), &descriptor); + Ok( + SourceSetContext::new(format!("tool:{}", extension.name), source_path, state) + .with_excluded_roots(vec![absolutize_path(&config.work_path)?]), + ) } fn absolutize_path(path: &Path) -> Result { @@ -223,30 +393,16 @@ fn extension_stage_detail(executor: &str, action: &str, extension: &ToolExtensio format!("[{executor}] {action} расширения {}", extension.name) } -fn commit_tool_extension_full_rescan( +fn commit_tool_extension_full_observation( context: &SourceSetContext, - work_path: &Path, - recover_storage: bool, + prepared: &analyzer::PreparedStateUpdate, ) -> Result<(), AppError> { - match analyzer::rescan_and_commit_full(context, work_path) { - Ok(()) => Ok(()), - Err(_error) if recover_storage && storage_needs_recovery(context, work_path) => { - let storage_path = context.storage_path(work_path); - remove_storage_path(&storage_path).map_err(|remove_error| { - AppError::Runtime(format!( - "failed to remove corrupt storage '{}': {remove_error}", - storage_path.display() - )) - })?; - analyzer::rescan_and_commit_full(context, work_path) - .map_err(|retry_error| AppError::Runtime(retry_error.to_string())) - } - Err(error) => Err(AppError::Runtime(error.to_string())), - } + analyzer::commit_full_observation(context, prepared) + .map_err(|error| AppError::Runtime(error.to_string())) } -fn storage_needs_recovery(context: &SourceSetContext, work_path: &Path) -> bool { - match HashStorage::new(context.storage_path(work_path)).current_generation() { +fn storage_needs_recovery(context: &SourceSetContext) -> bool { + match HashStorage::new(context.storage_path()).current_generation() { Err(StorageError::Recoverable { .. }) => true, Err(StorageError::Hard { reason, .. }) => { let reason = reason.to_ascii_lowercase(); @@ -256,28 +412,46 @@ fn storage_needs_recovery(context: &SourceSetContext, work_path: &Path) -> bool } } -fn remove_storage_path(path: &Path) -> std::io::Result<()> { - if !path.exists() { - return Ok(()); - } - - let metadata = std::fs::symlink_metadata(path)?; - if metadata.is_dir() { - std::fs::remove_dir_all(path) - } else { - std::fs::remove_file(path) - } -} - fn prepare_designer_source_extension( context: &ExecutionContext, config: &AppConfig, extension: &ToolExtensionConfig, source_path: &Path, utilities: &mut PlatformUtilities, + commit: &ToolSourceCommit<'_>, ) -> Result<(), AppError> { match config.builder { BuilderBackend::Designer => { + let state_lock = lock_designer_state(commit.context) + .map_err(|error| AppError::Runtime(error.to_string()))?; + recover_designer_state_with_lock(commit.context, &state_lock) + .map_err(|error| AppError::Runtime(error.to_string()))?; + cleanup_orphan_designer_transactions(commit.context, &state_lock) + .map_err(|error| AppError::Runtime(error.to_string()))?; + let private_cdfi = if commit.cdfi_seed == ToolCdfiSeedPolicy::Allow { + match inspect_private_cdfi(&commit.context.private_cdfi_path()) + .map_err(|error| AppError::Runtime(error.to_string()))? + { + PrivateCdfiState::Valid(cdfi) => Some(cdfi), + PrivateCdfiState::Missing | PrivateCdfiState::Corrupt(_) => None, + } + } else { + None + }; + let transaction = DesignerSourceTransaction::create( + source_path, + commit.context.excluded_roots(), + &commit.context.transactions_dir(), + private_cdfi + .as_ref() + .map_or(CdfiSeed::None, CdfiSeed::Validated), + ) + .map_err(|error| AppError::Runtime(error.to_string()))?; + if source_path == commit.context.path() { + transaction + .verify_snapshot(commit.prepared) + .map_err(|error| AppError::Runtime(error.to_string()))?; + } let binary = utilities .locate(UtilityType::V8) .map_err(AppError::from)? @@ -290,13 +464,20 @@ fn prepare_designer_source_extension( extension, "load", )?; + if let Some(error) = + interruption_before_safe_point(context, extension, "Designer tool extension load") + { + return Err(error); + } + require_designer_full_rebuild(commit.context) + .map_err(|error| AppError::Runtime(error.to_string()))?; log_tool_extension_stage( extension, "load", &extension_stage_detail("Конфигуратор", "Загрузка", extension), ); let load = dsl - .load_config_from_files_full(source_path, Some(&extension.name)) + .load_config_from_files_full(transaction.load_root(), Some(&extension.name)) .map_err(AppError::from)?; ensure_tool_extension_success("load", extension, &load)?; @@ -321,7 +502,17 @@ fn prepare_designer_source_extension( let update = dsl .update_db_cfg(Some(&extension.name)) .map_err(AppError::from)?; - ensure_tool_extension_success("update_db_cfg", extension, &update) + ensure_tool_extension_success("update_db_cfg", extension, &update)?; + commit_designer_state_with_lock( + commit.context, + &state_lock, + commit.prepared, + &transaction.load_root().join("ConfigDumpInfo.xml"), + ) + .map_err(|error| AppError::Runtime(error.to_string()))?; + transaction + .close() + .map_err(|error| AppError::Runtime(error.to_string())) } BuilderBackend::Ibcmd => { let binary = utilities @@ -633,9 +824,21 @@ fn successful_build_step( ok: true, message: Some(message), duration_ms, + receipt: crate::domain::sync_receipt::SyncReceipt::empty_applied(), } } +fn successful_build_step_with_receipt( + extension: &ToolExtensionConfig, + message: String, + duration_ms: u64, + receipt: SyncReceipt, +) -> BuildStep { + let mut step = successful_build_step(extension, message, duration_ms); + step.receipt = receipt; + step +} + fn skipped_build_step( extension: &ToolExtensionConfig, message: String, @@ -647,17 +850,149 @@ fn skipped_build_step( ok: true, message: Some(message), duration_ms, + receipt: crate::domain::sync_receipt::SyncReceipt::empty_skipped(), } } -fn failed_build_step(extension: &ToolExtensionConfig, message: String) -> BuildStep { +fn failed_build_step( + extension: &ToolExtensionConfig, + message: String, + receipt: SyncReceipt, +) -> BuildStep { BuildStep { source_set: format!("tool:{}", extension.name), mode: BuildMode::Full, ok: false, message: Some(message), duration_ms: 0, + receipt, + } +} + +fn applied_receipt( + requested: Vec, + processed: Vec, +) -> Result { + SyncReceipt::applied(requested, processed, vec![]) + .map_err(|error| AppError::Runtime(error.to_string())) +} + +fn tool_extension_failure( + extension: &ToolExtensionConfig, + error: AppError, + requested: Vec, +) -> ToolExtensionFailure { + let (error, receipt) = match SyncReceipt::failed(requested) { + Ok(receipt) => (error, receipt), + Err(receipt_error) => { + let combined = AppError::Runtime(format!( + "{error}; failed to construct exact failure receipt: {receipt_error}" + )); + (combined, SyncReceipt::empty_failed()) + } + }; + ToolExtensionFailure { + step: failed_build_step(extension, error.to_string(), receipt), + error, + } +} + +fn full_inventory_receipt( + context: &SourceSetContext, +) -> Result< + ( + Vec, + Vec, + analyzer::PreparedStateUpdate, + ), + AppError, +> { + let inventory = analyzer::managed_inventory(context) + .map_err(|error| AppError::Runtime(error.to_string()))?; + let requested = inventory + .requested + .iter() + .map(|change| { + SyncTarget::new( + &change.rel_path, + change.pre_hash.clone(), + change.post_hash.clone(), + ) + .map_err(|error| AppError::Runtime(error.to_string())) + }) + .collect::, _>>()?; + let requested_by_path = unique_tool_target_index(&requested)?; + let mut processed = inventory + .current + .into_iter() + .map(|file| { + let pre_hash = match requested_by_path.get(file.rel_path.as_str()) { + Some(target) => target.pre_hash().map(str::to_owned), + None => Some(file.hash.clone()), + }; + SyncTarget::new(file.rel_path, pre_hash, Some(file.hash)) + .map_err(|error| AppError::Runtime(error.to_string())) + }) + .collect::, _>>()?; + processed.extend( + requested + .iter() + .filter(|target| target.post_hash().is_none()) + .cloned(), + ); + Ok((requested, processed, inventory.prepared)) +} + +fn change_receipt( + changes: &[analyzer::FileChange], + prepared: &analyzer::PreparedStateUpdate, +) -> Result<(Vec, Vec), AppError> { + let requested = changes + .iter() + .map(|change| { + SyncTarget::new( + &change.rel_path, + change.pre_hash.clone(), + change.post_hash.clone(), + ) + .map_err(|error| AppError::Runtime(error.to_string())) + }) + .collect::, _>>()?; + let requested_by_path = unique_tool_target_index(&requested)?; + let mut processed = prepared + .snapshot + .iter() + .map(|file| { + let pre_hash = match requested_by_path.get(file.rel_path.as_str()) { + Some(target) => target.pre_hash().map(str::to_owned), + None => Some(file.hash.clone()), + }; + SyncTarget::new(&file.rel_path, pre_hash, Some(file.hash.clone())) + .map_err(|error| AppError::Runtime(error.to_string())) + }) + .collect::, _>>()?; + processed.extend( + requested + .iter() + .filter(|target| target.post_hash().is_none()) + .cloned(), + ); + Ok((requested, processed)) +} + +fn unique_tool_target_index( + targets: &[SyncTarget], +) -> Result, AppError> { + let mut by_path = std::collections::HashMap::with_capacity(targets.len()); + for target in targets { + if by_path.insert(target.path(), target).is_some() { + return Err(AppError::Runtime(format!( + "duplicate normalized path '{}' in tool extension inventory", + target.path() + ))); + } } + Ok(by_path) } fn recreate_directory(path: &Path) -> std::io::Result<()> { diff --git a/tests/cli_build.rs b/tests/cli_build.rs index c0409f8..96b00ba 100644 --- a/tests/cli_build.rs +++ b/tests/cli_build.rs @@ -22,7 +22,7 @@ fn write_build_script(path: &Path, fail_pattern: Option<&str>) { }) .unwrap_or_default(); let body = format!( - "args=\"$*\"\nout=\"\"\nprev=\"\"\nfor arg in \"$@\"; do\n if [ \"$prev\" = \"/Out\" ]; then out=\"$arg\"; fi\n prev=\"$arg\"\ndone\nif [ -n \"$out\" ]; then printf 'designer log for %s\\n' \"$args\" > \"$out\"; fi\n{}\nexit 0", + "args=\"$*\"\nout=\"\"\nload_root=\"\"\nprev=\"\"\nfor arg in \"$@\"; do\n if [ \"$prev\" = \"/Out\" ]; then out=\"$arg\"; fi\n if [ \"$prev\" = \"/LoadConfigFromFiles\" ]; then load_root=\"$arg\"; fi\n prev=\"$arg\"\ndone\nif [ -n \"$out\" ]; then printf 'designer log for %s\\n' \"$args\" > \"$out\"; fi\n{}\nif [ -n \"$load_root\" ]; then printf '%s\\n' '' > \"$load_root/ConfigDumpInfo.xml\"; fi\nexit 0", pattern_branch ); write_script(path, &body); @@ -212,6 +212,27 @@ fn setup_project() -> (tempfile::TempDir, PathBuf, PathBuf, PathBuf) { (dir, config_path, binary_path, work_path) } +fn prime_designer_state_and_modify_main(config_path: &Path) { + let prime = v8_runner_command() + .args([ + "--config", + &config_path.display().to_string(), + "build", + "--full-rebuild", + ]) + .output() + .expect("prime Designer state"); + assert!(prime.status.success(), "prime build failed"); + fs::write( + config_path + .parent() + .expect("config parent") + .join("project/main/Catalogs.Items/ObjectModule.bsl"), + "procedure Test() // changed\nendprocedure", + ) + .expect("modify main source"); +} + fn setup_ibcmd_project() -> ( tempfile::TempDir, PathBuf, @@ -462,6 +483,7 @@ fn build_text_failure_does_not_print_success_footer() { #[test] fn build_text_stdout_includes_action_logs() { let (_dir, config_path, _binary_path, _work_path) = setup_project(); + prime_designer_state_and_modify_main(&config_path); let output = v8_runner_command() .args([ @@ -488,6 +510,7 @@ fn build_text_stdout_includes_action_logs() { #[test] fn build_text_highlights_timeline_detail_prefixes() { let (_dir, config_path, _binary_path, _work_path) = setup_project(); + prime_designer_state_and_modify_main(&config_path); let output = v8_runner_command() .args(["--config", &config_path.display().to_string(), "build"]) @@ -671,7 +694,7 @@ fn build_edt_text_interleaves_export_stage_after_edt_log() { assert!(stdout.contains("│ ✓ completed")); assert!(stdout.contains("[ibcmd] Загрузка в базу")); assert!(stdout.contains("[ibcmd] Применение изменений")); - assert!(stdout.contains("│ ✓ full load selected by partial-load rules")); + assert!(stdout.contains("│ ✓ fallback to full load after recoverable change-detection issue")); assert_eq!(stdout.matches("● configuration").count(), 1); let ibcmd_calls = fs::read_to_string(ibcmd_calls_log).expect("ibcmd calls"); @@ -748,6 +771,7 @@ fn build_text_groups_tool_extension_stages_under_single_build_node() { #[test] fn build_json_writes_action_log_file_without_polluting_stdout() { let (_dir, config_path, _binary_path, work_path) = setup_project(); + prime_designer_state_and_modify_main(&config_path); let output = v8_runner_command() .args([ diff --git a/tests/cli_dump.rs b/tests/cli_dump.rs index d96260d..21ec35b 100644 --- a/tests/cli_dump.rs +++ b/tests/cli_dump.rs @@ -2,6 +2,7 @@ mod support; +use std::collections::BTreeMap; use std::fs; use std::path::{Path, PathBuf}; @@ -21,7 +22,7 @@ fn write_ibcmd_script(path: &Path, calls_log: &Path, fail_pattern: Option<&str>) }) .unwrap_or_default(); let body = format!( - "args=\"$*\"\nprintf '%s\\n' \"$args\" >> \"{}\"\n{}\nmkdir -p \"$(printf '%s' \"$args\" | awk '{{print $NF}}')\"\nexit 0", + "args=\"$*\"\ntarget=\"$(printf '%s' \"$args\" | awk '{{print $NF}}')\"\nprintf '%s\\n' \"$args\" >> \"{}\"\n{}\nmkdir -p \"$target\"\nprintf '\\n' > \"$target/ConfigDumpInfo.xml\"\nexit 0", calls_log.display(), pattern_branch ); @@ -30,7 +31,7 @@ fn write_ibcmd_script(path: &Path, calls_log: &Path, fail_pattern: Option<&str>) fn write_designer_dump_script_for_edt(path: &Path, calls_log: &Path) { let body = format!( - "args=\"$*\"\nout=\"\"\ntarget=\"\"\nprev=\"\"\nfor arg in \"$@\"; do\n if [ \"$prev\" = \"/Out\" ]; then out=\"$arg\"; fi\n if [ \"$prev\" = \"/DumpConfigToFiles\" ]; then target=\"$arg\"; fi\n prev=\"$arg\"\ndone\nif [ -n \"$out\" ]; then printf 'designer log for %s\\n' \"$args\" > \"$out\"; fi\nprintf '%s\\n' \"$args\" >> \"{}\"\nmkdir -p \"$target\"\nprintf '\\n' > \"$target/Configuration.xml\"\nexit 0", + "args=\"$*\"\nout=\"\"\ntarget=\"\"\nprev=\"\"\nfor arg in \"$@\"; do\n if [ \"$prev\" = \"/Out\" ]; then out=\"$arg\"; fi\n if [ \"$prev\" = \"/DumpConfigToFiles\" ]; then target=\"$arg\"; fi\n prev=\"$arg\"\ndone\nif [ -n \"$out\" ]; then printf 'designer log for %s\\n' \"$args\" > \"$out\"; fi\nprintf '%s\\n' \"$args\" >> \"{}\"\nmkdir -p \"$target\"\nprintf '\\n' > \"$target/Configuration.xml\"\nprintf '\\n' > \"$target/ConfigDumpInfo.xml\"\nexit 0", calls_log.display() ); write_script(path, &body); @@ -38,7 +39,7 @@ fn write_designer_dump_script_for_edt(path: &Path, calls_log: &Path) { fn write_designer_partial_dump_script(path: &Path, captured_list: &Path) { let body = format!( - "list_file=\"\"\ntarget=\"\"\nprevious=\"\"\nfor argument in \"$@\"; do\n if [ \"$previous\" = \"-listFile\" ]; then list_file=\"$argument\"; fi\n if [ \"$previous\" = \"/DumpConfigToFiles\" ]; then target=\"$argument\"; fi\n previous=\"$argument\"\ndone\ncp \"$list_file\" \"{}\"\nmkdir -p \"$target\"\nprintf '\\n' > \"$target/Configuration.xml\"\nexit 0", + "list_file=\"\"\ntarget=\"\"\nprevious=\"\"\nfor argument in \"$@\"; do\n if [ \"$previous\" = \"-listFile\" ]; then list_file=\"$argument\"; fi\n if [ \"$previous\" = \"/DumpConfigToFiles\" ]; then target=\"$argument\"; fi\n previous=\"$argument\"\ndone\nif [ -n \"$list_file\" ]; then cp \"$list_file\" \"{}\"; fi\nmkdir -p \"$target\"\nprintf '\\n' > \"$target/Configuration.xml\"\nprintf '\\n' > \"$target/ConfigDumpInfo.xml\"\nexit 0", captured_list.display() ); write_script(path, &body); @@ -154,8 +155,6 @@ fn setup_project() -> ( fs::create_dir_all(base_path.join("main")).expect("main"); fs::create_dir_all(&work_path).expect("work"); - fs::write(base_path.join("main").join("old.txt"), "old").expect("old"); - write_ibcmd_script(&binary_path, &calls_log, None); write_config(&config_path, &base_path, &work_path, &binary_path); @@ -226,8 +225,6 @@ fn setup_edt_project() -> ( fs::create_dir_all(base_path.join("main")).expect("main"); fs::create_dir_all(&work_path).expect("work"); write_edt_configuration_source(&base_path.join("main"), "BaseProject"); - fs::write(base_path.join("main").join("old.txt"), "old").expect("old"); - write_designer_dump_script_for_edt(&platform_path, &designer_calls); write_edt_import_script(&edt_path, &edt_calls); write_edt_dump_config( @@ -250,6 +247,63 @@ fn setup_edt_project() -> ( ) } +fn bootstrap_full(config_path: &Path) { + let output = v8_runner_command() + .args([ + "--config", + &config_path.display().to_string(), + "--json-message", + "dump", + "--mode", + "full", + "--source-set", + "main", + ]) + .output() + .expect("bootstrap dump"); + assert!( + output.status.success(), + "bootstrap stdout:\n{}\nbootstrap stderr:\n{}", + String::from_utf8_lossy(&output.stdout), + String::from_utf8_lossy(&output.stderr) + ); +} + +fn snapshot_runtime_generations(root: &Path) -> BTreeMap> { + fn visit(root: &Path, current: &Path, files: &mut BTreeMap>) { + if !current.exists() { + return; + } + let mut entries = fs::read_dir(current) + .expect("read snapshot directory") + .collect::, _>>() + .expect("snapshot entries"); + entries.sort_by_key(fs::DirEntry::file_name); + for entry in entries { + let path = entry.path(); + let file_type = entry.file_type().expect("snapshot file type"); + if file_type.is_dir() { + visit(root, &path, files); + } else if file_type.is_file() { + let relative = path + .strip_prefix(root) + .expect("relative snapshot path") + .to_path_buf(); + if relative + .components() + .any(|component| component.as_os_str() == "generations") + { + files.insert(relative, fs::read(path).expect("snapshot file")); + } + } + } + } + + let mut files = BTreeMap::new(); + visit(root, root, &mut files); + files +} + #[test] fn dump_ibcmd_full_json_success() { let (_dir, config_path, _binary_path, work_path, base_path, calls_log) = setup_project(); @@ -271,6 +325,11 @@ fn dump_ibcmd_full_json_success() { assert!(output.status.success()); let payload: Value = serde_json::from_slice(&output.stdout).expect("json"); assert_eq!(payload["ok"], true); + assert_eq!(payload["data"]["receipt"]["status"], "applied"); + assert_eq!( + payload["data"]["receipt"]["requested"], + serde_json::json!([]) + ); let calls = fs::read_to_string(calls_log).expect("calls"); assert!(calls.contains("--force")); assert_ibcmd_data_path(&calls, &work_path); @@ -278,7 +337,7 @@ fn dump_ibcmd_full_json_success() { } #[test] -fn dump_edt_full_json_success_updates_designer_mirror_and_edt_target() { +fn dump_edt_full_json_success_updates_runtime_designer_baseline_and_edt_target() { let ( _dir, config_path, @@ -315,26 +374,22 @@ fn dump_edt_full_json_success_updates_designer_mirror_and_edt_target() { assert_eq!(payload["command"], "dump"); assert_eq!( payload["data"]["target_path"], - base_path.join("main").display().to_string() + fs::canonicalize(base_path.join("main")) + .expect("canonical target") + .display() + .to_string() ); + assert_eq!(payload["data"]["receipt"]["status"], "applied"); assert_native_edt_project(&base_path.join("main")); - assert!(!base_path.join("main").join("old.txt").exists()); - assert!(work_path - .join("designer") - .join("main") - .join("Configuration.xml") - .exists()); + let runtime_generations = snapshot_runtime_generations(&work_path); + assert!(runtime_generations.keys().any(|path| { + path.ends_with("ib-baseline/edt-platform-designer/files/Configuration.xml") + })); let designer_calls = fs::read_to_string(designer_calls).expect("designer calls"); let edt_calls = fs::read_to_string(edt_calls).expect("edt calls"); - assert!(designer_calls.contains(work_path.join("designer").display().to_string().as_str())); - assert!(edt_calls.contains( - work_path - .join("designer/main") - .display() - .to_string() - .as_str() - )); + assert!(designer_calls.contains(work_path.display().to_string().as_str())); + assert!(edt_calls.contains(work_path.display().to_string().as_str())); } #[test] @@ -378,7 +433,8 @@ fn dump_text_success_is_compact_and_keeps_output_visible() { #[test] fn dump_ibcmd_incremental_json_success() { let (_dir, config_path, _binary_path, work_path, base_path, calls_log) = setup_project(); - fs::remove_dir_all(base_path.join("main")).expect("remove target"); + bootstrap_full(&config_path); + fs::write(&calls_log, []).expect("clear calls"); let output = v8_runner_command() .args([ @@ -397,15 +453,19 @@ fn dump_ibcmd_incremental_json_success() { assert!(output.status.success()); let payload: Value = serde_json::from_slice(&output.stdout).expect("json"); assert_eq!(payload["ok"], true); + assert_eq!(payload["data"]["receipt"]["status"], "applied"); let calls = fs::read_to_string(calls_log).expect("calls"); assert!(calls.contains("--sync")); assert_ibcmd_data_path(&calls, &work_path); - assert!(calls.contains(base_path.join("main").display().to_string().as_str())); + assert!(calls.contains(work_path.display().to_string().as_str())); + assert!(!calls.contains(base_path.join("main").display().to_string().as_str())); } #[test] fn dump_ibcmd_partial_json_success_uses_degraded_fallback() { let (_dir, config_path, _binary_path, work_path, _base_path, calls_log) = setup_project(); + bootstrap_full(&config_path); + fs::write(&calls_log, []).expect("clear calls"); let output = v8_runner_command() .args([ @@ -428,6 +488,7 @@ fn dump_ibcmd_partial_json_success_uses_degraded_fallback() { let data = &payload["data"]; assert_eq!(payload["ok"], true); assert_eq!(data["mode"], "PARTIAL"); + assert_eq!(data["receipt"]["status"], "applied"); assert!(data["message"] .as_str() .expect("message") @@ -440,6 +501,7 @@ fn dump_ibcmd_partial_json_success_uses_degraded_fallback() { #[test] fn dump_text_warning_shows_degraded_fallback_reason() { let (_dir, config_path, _binary_path, _work_path, _base_path, _calls_log) = setup_project(); + bootstrap_full(&config_path); let output = v8_runner_command() .args([ @@ -466,6 +528,7 @@ fn dump_text_warning_shows_degraded_fallback_reason() { #[test] fn dump_ibcmd_partial_failure_keeps_partial_mode_and_warning() { let (_dir, config_path, binary_path, _work_path, _base_path, calls_log) = setup_project(); + bootstrap_full(&config_path); write_ibcmd_script(&binary_path, &calls_log, Some("--sync")); let output = v8_runner_command() @@ -490,6 +553,11 @@ fn dump_ibcmd_partial_failure_keeps_partial_mode_and_warning() { let data = &payload["data"]; assert_eq!(payload["ok"], false); assert_eq!(data["mode"], "PARTIAL"); + assert_eq!(data["receipt"]["status"], "failed"); + assert_eq!(data["receipt"]["requested"], serde_json::json!([])); + assert_eq!(data["receipt"]["processed"], serde_json::json!([])); + assert_eq!(data["receipt"]["skipped"], serde_json::json!([])); + assert_eq!(data["receipt"]["conflicted"], serde_json::json!([])); assert!(data["message"] .as_str() .expect("message") @@ -510,6 +578,7 @@ fn dump_designer_partial_json_normalizes_colon_selector_and_reports_both_forms() .join("partial-list.txt"); write_designer_partial_dump_script(&designer_binary, &captured_list); write_designer_config(&config_path, &work_path, &designer_binary); + bootstrap_full(&config_path); let output = v8_runner_command() .args([ @@ -567,6 +636,104 @@ fn dump_text_failure_shows_error_message() { assert!(stdout.contains("exit code 17")); } +#[test] +fn dump_json_conflict_preserves_local_file_and_runtime_state() { + let (_dir, config_path, _binary_path, work_path, base_path, _calls_log) = setup_project(); + bootstrap_full(&config_path); + let local_path = base_path.join("main/old.txt"); + fs::write(&local_path, "old").expect("local file"); + let state_before = snapshot_runtime_generations(&work_path); + + let output = v8_runner_command() + .args([ + "--config", + &config_path.display().to_string(), + "--json-message", + "dump", + "--mode", + "full", + "--source-set", + "main", + ]) + .output() + .expect("run command"); + + assert!(!output.status.success()); + assert_eq!(output.status.code(), Some(3)); + let payload: Value = serde_json::from_slice(&output.stdout).expect("json"); + let receipt = &payload["data"]["receipt"]; + assert_eq!(payload["error"]["kind"], "runtime"); + assert_eq!(receipt["status"], "conflict"); + assert_eq!(receipt["processed"], serde_json::json!([])); + assert_eq!(receipt["requested"], receipt["conflicted"]); + assert_eq!(receipt["requested"].as_array().expect("requested").len(), 1); + assert_eq!(receipt["requested"][0]["path"], "old.txt"); + assert!(receipt["requested"][0]["preHash"].is_string()); + assert_eq!(receipt["requested"][0]["postHash"], Value::Null); + assert_eq!( + fs::read(&local_path).expect("local file after conflict"), + b"old" + ); + assert_eq!(snapshot_runtime_generations(&work_path), state_before); +} + +#[test] +fn dump_json_incremental_and_partial_conflicts_preserve_source_and_runtime_state() { + for (mode, objects) in [ + ("incremental", Vec::<&str>::new()), + ("partial", vec!["--object", "Catalog.Items"]), + ] { + let (_dir, config_path, _binary_path, work_path, base_path, calls_log) = setup_project(); + bootstrap_full(&config_path); + fs::write(&calls_log, []).expect("clear calls"); + let local_path = base_path.join("main/old.txt"); + fs::write(&local_path, "old").expect("local file"); + let state_before = snapshot_runtime_generations(&work_path); + + let mut args = vec![ + "--config", + config_path.to_str().expect("config path"), + "--json-message", + "dump", + "--mode", + mode, + "--source-set", + "main", + ]; + args.extend(objects); + let output = v8_runner_command() + .args(args) + .output() + .expect("run command"); + + assert!(!output.status.success(), "{mode} dump must conflict"); + assert_eq!(output.status.code(), Some(3), "unexpected {mode} exit code"); + let payload: Value = serde_json::from_slice(&output.stdout).expect("json"); + let receipt = &payload["data"]["receipt"]; + assert_eq!(payload["data"]["mode"], mode.to_uppercase()); + assert_eq!(receipt["status"], "conflict"); + assert_eq!(receipt["processed"], serde_json::json!([])); + assert_eq!(receipt["requested"], receipt["conflicted"]); + assert_eq!(receipt["requested"][0]["path"], "old.txt"); + assert_eq!( + fs::read(&local_path).expect("local file after conflict"), + b"old", + "{mode} dump changed local source" + ); + assert_eq!( + snapshot_runtime_generations(&work_path), + state_before, + "{mode} dump advanced runtime state after conflict" + ); + assert!( + fs::read_to_string(&calls_log) + .expect("calls") + .contains("--sync"), + "{mode} dump did not exercise the incremental shadow branch" + ); + } +} + #[test] fn dump_ibcmd_full_server_connection_passes_dbms_and_infobase_credentials() { let (_dir, config_path, _binary_path, work_path, _base_path, calls_log) = setup_project(); diff --git a/tests/cli_test.rs b/tests/cli_test.rs index afa0928..1595107 100644 --- a/tests/cli_test.rs +++ b/tests/cli_test.rs @@ -60,7 +60,7 @@ fn write_build_script(path: &Path, calls_log: &Path, fail: bool) { "" }; let body = format!( - "args=\"$*\"\nout=\"\"\nprev=\"\"\nfor arg in \"$@\"; do\n if [ \"$prev\" = \"/Out\" ]; then out=\"$arg\"; fi\n prev=\"$arg\"\ndone\nprintf '%s\\n' \"$args\" >> '{}'\nif [ -n \"$out\" ]; then printf 'build /P secret\\n' > \"$out\"; fi\n{}\nexit 0", + "args=\"$*\"\nout=\"\"\nload_root=\"\"\nprev=\"\"\nfor arg in \"$@\"; do\n if [ \"$prev\" = \"/Out\" ]; then out=\"$arg\"; fi\n if [ \"$prev\" = \"/LoadConfigFromFiles\" ]; then load_root=\"$arg\"; fi\n prev=\"$arg\"\ndone\nprintf '%s\\n' \"$args\" >> '{}'\nif [ -n \"$out\" ]; then printf 'build /P secret\\n' > \"$out\"; fi\n{}\nif [ -n \"$load_root\" ]; then printf '%s\\n' '' > \"$load_root/ConfigDumpInfo.xml\"; fi\nexit 0", calls_log.display(), fail_branch ); @@ -642,6 +642,16 @@ fn test_accepts_explicit_client_mode_for_vanessa_and_yaxunit() { 5, None, ); + let prime = v8_runner_command() + .args([ + "--config", + &config_path.display().to_string(), + "build", + "--full-rebuild", + ]) + .output() + .expect("prime private Designer state"); + assert!(prime.status.success(), "prime build failed"); write_script( &dir.path().join("platform").join("bin").join("1cv8"), &format!(