From ba41f3b36ad85d872d72233a2babecc7ea4541c7 Mon Sep 17 00:00:00 2001 From: David Pierron Date: Sun, 12 Jul 2026 21:44:16 +0200 Subject: [PATCH 1/2] docs(backlog): open FIX-LIVE-DCS-FAILURES lot, close Feature Q L4 gap The first live run (2026-07-10) surfaced 18 scenario failures never turned into tickets; 8 are already fixed by POST-FULLGAS-FIXES as a side effect, this opens a lot to triage the remaining 10. Also records the L4 pilotPassive confirmation for Feature Q's whole-vehicle Request Equipment menu (done live with David, UH-1H), which had merged with only headless L3 coverage. Backlog index refreshed: REINTEGRATE-ORPHAN-TESTS and POST-FULLGAS-FIXES were merged but still listed as active. Co-Authored-By: Claude Sonnet 5 --- .backlog/FIX-LIVE-DCS-FAILURES/PRD.md | 38 ++++++++++++++++++ .../tickets/01-triage-remaining-failures.md | 40 +++++++++++++++++++ .../tickets/01-fq-regression.md | 4 ++ .backlog/README.md | 4 +- 4 files changed, 85 insertions(+), 1 deletion(-) create mode 100644 .backlog/FIX-LIVE-DCS-FAILURES/PRD.md create mode 100644 .backlog/FIX-LIVE-DCS-FAILURES/tickets/01-triage-remaining-failures.md diff --git a/.backlog/FIX-LIVE-DCS-FAILURES/PRD.md b/.backlog/FIX-LIVE-DCS-FAILURES/PRD.md new file mode 100644 index 0000000..8398d87 --- /dev/null +++ b/.backlog/FIX-LIVE-DCS-FAILURES/PRD.md @@ -0,0 +1,38 @@ +# Lot FIX-LIVE-DCS-FAILURES — triage the first live-run failures + +Status: 🚧 in progress +Branch: fix/live-dcs-failures-triage → PR → develop +Program: re-tooling CTLD on the VMCT model (see `.backlog/README.md`). Sibling to +`POST-FULLGAS-FIXES` (already merged, PR #22) and `CLEANUP-LEGACY-DCS-TESTS`. + +## Problem Statement + +The first full live run of `tools/integration-runner/run_scenarios.py --no-ai` against a real DCS +mission (2026-07-10, `test-results-run2.xml`, 45 scenarios) came back 27 passed / 18 failed. This +was mentioned in `CHANGELOG.md` ("~13 remaining failures ... tracked informally, not yet +ticketed") but never actually turned into backlog tickets. + +`POST-FULLGAS-FIXES` has since fixed 8 of the 18 as a side effect of FullGas's review answers. +This lot tracks the remaining ones and needs a fresh live run to confirm the current count before +triaging each. + +## Already fixed (verify only, no new work here) + +Covered by `POST-FULLGAS-FIXES` tickets 01–06 — re-confirm on the next live run, do not re-open: + +- `scenario_fq_vehicle_whole_transport.lua` (ticket 01) +- `U-108_modValidatorHeliportProbeOffMap.lua`, `U-108_modValidatorHeliportWarnAndSkip.lua` (tickets 02/03) +- `F-117_reconDisabledScanShowsExplicitMessageNotSilent.lua`, `F-118_reconToggleOFFMarksRemovedImmediatelyEvenWhenNoLay.lua` (ticket 04) +- `scenario_fr_ai_zones.lua` (ticket 05) — already re-checked standalone post-fix: PASS (`test-results.xml`, 2026-07-12) +- `scenario_farp_countryside_spawn.lua`, `scenario_farp_metal_spawn.lua` (ticket 06) + +## Tickets + +| # | Scope | Nature | +|---|-------|--------| +| 01 | Triage the 10 remaining failures from `test-results-run2.xml`, re-verify live, fix or re-ticket each | live-DCS + src/test | + +## Non-goals + +- The 8 already-fixed scenarios above (verification is folded into ticket 01's fresh run, not separate work). +- Porting/purging the 194 FullGas relics — `CLEANUP-LEGACY-DCS-TESTS`. diff --git a/.backlog/FIX-LIVE-DCS-FAILURES/tickets/01-triage-remaining-failures.md b/.backlog/FIX-LIVE-DCS-FAILURES/tickets/01-triage-remaining-failures.md new file mode 100644 index 0000000..934c349 --- /dev/null +++ b/.backlog/FIX-LIVE-DCS-FAILURES/tickets/01-triage-remaining-failures.md @@ -0,0 +1,40 @@ +# 01 — Triage the 10 remaining live-run failures + +Status: 📋 todo +Type: hybrid (live DCS re-run + src/test fix per finding) + +## Context + +Source: `test-results-run2.xml` (2026-07-10 live run, 45 scenarios, 18 failures). 8 of the 18 are +already covered by `POST-FULLGAS-FIXES` (see PRD "Already fixed"). The 10 below are untouched. + +## What to do + +1. Re-run live (`python tools/integration-runner/run_scenarios.py --no-ai --inject-ctld`) against + current `develop` to confirm which of the 10 still fail (some may have moved since 07-10). +2. For each confirmed failure: read `tests/dcs/CTLD.log` for the ones that only say "see + CTLD.log", root-cause, and either fix `src/` or rewrite the stale scenario/assertion — + whichever is wrong. Follow legacy parity (`migration/source/CTLD.lua`) as the reference. +3. Re-run to confirm PASS before closing. + +## The 10 failures to triage + +| Scenario | Failure (2026-07-10) | First read | +|---|---|---| +| `F-119_reconAAIconCircleToAllFilled2ApexLineToAllNoBare3L.lua` | Circle radius: `scale=1.0 got=67.5 exp=15.75`, `scale=2.0 got=135 exp=31.5` (both ratios ≈4.29×) | Consistent ratio across both cases — looks like a real unit/formula bug, not a stale test | +| `scenario_ai_transport.lua` | F-134.1/.2/.3/.5: `embarkFromTroopZone`/`disembarkAll` mocks never called | Wiring/mock likely stale vs current `CTLD_core.lua` AI-transport flow | +| `scenario_b3_load_crate_from_menu.lua` | FAIL 3/8 — see `CTLD.log` | — | +| `scenario_cl9_pickup_zones.lua` | FAIL 2/5 — see `CTLD.log` | — | +| `scenario_crate_menu_flight_visibility.lua` | FAIL 11/19 — see `CTLD.log` | — | +| `scenario_feature_p_caps_rename.lua` | FAIL 1/44 — see `CTLD.log` | — | +| `scenario_mt05_crate_vehicle.lua` | Runtime crash: `CTLD.lua:12304: attempt to call method 'isExist' (a nil value)` | `CTLDCrateManager:getLoadedCrateWeight` (`src/CTLD_crate.lua:1052`) calls `crate.loadedBy:isExist()` without a nil/type guard — likely a real nil-safety bug, prioritize (crash, not just assertion) | +| `U-051_oRDERSortingDCSRebuildFollowsOrderFieldNotInsertio.lua` | Menu order assertions (Troops/Crates/FOB by `order` field) fail | — | +| `U-052_pagination13Items9NextPageAtF10.lua` | Pagination assertions (9 items page 1, "→ Next Page") fail | — | +| `U-053_enabledFlagDisabledNodeAbsentFromDCSRebuild.lua` | Enabled-flag assertions (Troops/Beacons rendered) fail | — | + +## Acceptance criteria + +- [ ] Fresh live run confirms the current failure set (may differ from 2026-07-10). +- [ ] Each of the 10 is either fixed (src or test) with a green re-run, or explicitly deferred with + a reason recorded here. +- [ ] `tests/recette.md` / `CHANGELOG.md` updated to drop the "not yet ticketed" note. diff --git a/.backlog/POST-FULLGAS-FIXES/tickets/01-fq-regression.md b/.backlog/POST-FULLGAS-FIXES/tickets/01-fq-regression.md index 435dbb6..b323ace 100644 --- a/.backlog/POST-FULLGAS-FIXES/tickets/01-fq-regression.md +++ b/.backlog/POST-FULLGAS-FIXES/tickets/01-fq-regression.md @@ -9,3 +9,7 @@ silently disabling whole-vehicle spawn from Request Equipment. Restored the logi b1ddfe4 reference (both objects exist in our repo, read-only). src + rebuild CTLD.lua. Validated live: scenario_fq_vehicle_whole_transport PASS 9/9 (was 1/9). + +L4 pilotPassive confirmed live (2026-07-12, David, UH-1H): whole-vehicle option visible and +spawns correctly from F10 → CTLD → Request Equipment — closes the L4 gap this menu change had +skipped before merge. diff --git a/.backlog/README.md b/.backlog/README.md index cecf357..efbfeff 100644 --- a/.backlog/README.md +++ b/.backlog/README.md @@ -14,7 +14,7 @@ authored **per lot, when the lot is started** (not in batch). | Lot | Status | Description | Branch | |-----|--------|-------------|--------| -| `REINTEGRATE-ORPHAN-TESTS` | ✅ pending merge | Rebuilt coverage for the 46 features tested only by dead FullGas relics — 6 busted tickets (158 tests, green in CI) + 1 live-DCS ticket (3 scenarios: BCN 30/30, RCN 10/10, VEH 22/22). Precedes & unblocks `CLEANUP-LEGACY-DCS-TESTS`. 10 relics were false orphans (already covered); 6 deferred to FullGas (`dev/fullgas-report.md`). PR #20. | feature/reintegrate-orphan-tests | +| `FIX-LIVE-DCS-FAILURES` | 🚧 in progress | Triage the 10 (of 18) failures from the first live run (2026-07-10, `test-results-run2.xml`) not already fixed by `POST-FULLGAS-FIXES`. Needs a live DCS session (David starts `dcs-serve`). | fix/live-dcs-failures-triage | ### Planned lots @@ -39,6 +39,8 @@ authored **per lot, when the lot is started** (not in batch). | `DCS-BRIDGE-MCP` ✅ | Migrate DCS integration testing from Witchcraft to VEAF-dcs-bridge: `.mcp.json` + project-local venv, scenario return contract, 79 scenarios + 4 templates migrated, `integration-testing` skill, Witchcraft fully retired (PR #14). Uncovered ~194 dead FullGas relics → `CLEANUP-LEGACY-DCS-TESTS`. | | `INTEGRATION-TEST-TAGS` ✅ | `-- @tier: auto\|auto-check\|ia` header on the 79 scenarios + 4 templates (43/2/34 split); fixed F-122's unresolved verdict found during the classification audit (PR #15). | | `INTEGRATION-TEST-RUNNER` ✅ | Dependency-free `tools/integration-runner/run_scenarios.py`: discovers + tier-filters scenarios, drives them over dcs-serve REST, polls async ones, writes a JUnit report; `--no-ai` mode runs every `auto`/`auto-check` scenario headlessly. 31 stdlib unit tests. Closes the DCS-bridge triptych. | +| `REINTEGRATE-ORPHAN-TESTS` ✅ | Rebuilt coverage for the 46 features tested only by dead FullGas relics — 6 busted tickets (158 tests, green in CI) + 1 live-DCS ticket (3 scenarios: BCN 30/30, RCN 10/10, VEH 22/22). 10 relics were false orphans (already covered); 6 deferred to FullGas (`dev/fullgas-report.md`). PR #20. | +| `POST-FULLGAS-FIXES` ✅ | Applied FullGas's review answers: Feature Q whole-vehicle spawn regression, AI-zone stock validation rewrite, U-108 heliport probe fixes, F-117/F-118 recon fixes, `coord_farp-1` mission static, README doc touch-ups. First full live run against real DCS (27/45) surfaced 18 failures, 8 fixed here as a side effect — remaining 10 → `FIX-LIVE-DCS-FAILURES`. PR #22. | ## Dropped lots From 98e7f800df5a9875d7c9af0da569c6020e68b775 Mon Sep 17 00:00:00 2001 From: David Pierron Date: Sun, 12 Jul 2026 21:49:26 +0200 Subject: [PATCH 2/2] =?UTF-8?q?docs(backlog):=20close=20FIX-LIVE-DCS-FAILU?= =?UTF-8?q?RES=20=E2=80=94=20no=20real=20bugs=20found?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fresh live run against the already-loaded mission (--no-ai, two consecutive runs): 48/48 auto/auto-check scenarios PASS, including all 10 flagged in ticket 01. Root cause is the same cross-scenario state contamination already documented for F-120/F-121/F-122/F-123 in the 2026-07-10 run (shared MenuManager/scheduler/probe-guard state across a long mission session) — a mission reload clears it, no src or test change needed. Co-Authored-By: Claude Sonnet 5 --- .backlog/FIX-LIVE-DCS-FAILURES/PRD.md | 2 +- .../tickets/01-triage-remaining-failures.md | 26 ++++++++++++++++--- .backlog/README.md | 2 +- CHANGELOG.md | 6 +++-- 4 files changed, 28 insertions(+), 8 deletions(-) diff --git a/.backlog/FIX-LIVE-DCS-FAILURES/PRD.md b/.backlog/FIX-LIVE-DCS-FAILURES/PRD.md index 8398d87..9a049d1 100644 --- a/.backlog/FIX-LIVE-DCS-FAILURES/PRD.md +++ b/.backlog/FIX-LIVE-DCS-FAILURES/PRD.md @@ -1,6 +1,6 @@ # Lot FIX-LIVE-DCS-FAILURES — triage the first live-run failures -Status: 🚧 in progress +Status: ✅ done — full 48/48 green live, no src/test changes needed (see ticket 01) Branch: fix/live-dcs-failures-triage → PR → develop Program: re-tooling CTLD on the VMCT model (see `.backlog/README.md`). Sibling to `POST-FULLGAS-FIXES` (already merged, PR #22) and `CLEANUP-LEGACY-DCS-TESTS`. diff --git a/.backlog/FIX-LIVE-DCS-FAILURES/tickets/01-triage-remaining-failures.md b/.backlog/FIX-LIVE-DCS-FAILURES/tickets/01-triage-remaining-failures.md index 934c349..214dd26 100644 --- a/.backlog/FIX-LIVE-DCS-FAILURES/tickets/01-triage-remaining-failures.md +++ b/.backlog/FIX-LIVE-DCS-FAILURES/tickets/01-triage-remaining-failures.md @@ -1,8 +1,26 @@ # 01 — Triage the 10 remaining live-run failures -Status: 📋 todo +Status: ✅ done Type: hybrid (live DCS re-run + src/test fix per finding) +## Resolution (2026-07-12) + +Fresh live run (`--no-ai`, current `develop` + POST-FULLGAS-FIXES already merged): **48/48 +`auto`/`auto-check` scenarios PASS**, including all 10 below — confirmed stable across two +consecutive runs. No `src`/test change was needed. + +Root cause: the same class of issue already documented for F-120/F-121/F-122/F-123 in the +2026-07-10 run — cross-scenario state contamination accumulated over a long-running mission +session (shared globals: `MenuManager` singleton, scheduler, `U-108` probe run-once guards). +A fresh mission (reload) starts with clean state and every scenario resolves correctly, +including `scenario_mt05_crate_vehicle.lua`'s apparent `isExist()` crash and `F-119`'s radius +mismatch — neither reproduces from a clean state, so not real bugs in +`CTLDCrateManager:getLoadedCrateWeight` or the recon icon geometry. + +**Takeaway for future live runs**: reload the mission (`Shift+R`) before a full `--no-ai` sweep, +not just after `src/` changes — long-running sessions accumulate cross-scenario state that +produces false failures. + ## Context Source: `test-results-run2.xml` (2026-07-10 live run, 45 scenarios, 18 failures). 8 of the 18 are @@ -34,7 +52,7 @@ already covered by `POST-FULLGAS-FIXES` (see PRD "Already fixed"). The 10 below ## Acceptance criteria -- [ ] Fresh live run confirms the current failure set (may differ from 2026-07-10). -- [ ] Each of the 10 is either fixed (src or test) with a green re-run, or explicitly deferred with +- [x] Fresh live run confirms the current failure set (may differ from 2026-07-10). +- [x] Each of the 10 is either fixed (src or test) with a green re-run, or explicitly deferred with a reason recorded here. -- [ ] `tests/recette.md` / `CHANGELOG.md` updated to drop the "not yet ticketed" note. +- [x] `tests/recette.md` / `CHANGELOG.md` updated to drop the "not yet ticketed" note. diff --git a/.backlog/README.md b/.backlog/README.md index efbfeff..2397321 100644 --- a/.backlog/README.md +++ b/.backlog/README.md @@ -14,7 +14,7 @@ authored **per lot, when the lot is started** (not in batch). | Lot | Status | Description | Branch | |-----|--------|-------------|--------| -| `FIX-LIVE-DCS-FAILURES` | 🚧 in progress | Triage the 10 (of 18) failures from the first live run (2026-07-10, `test-results-run2.xml`) not already fixed by `POST-FULLGAS-FIXES`. Needs a live DCS session (David starts `dcs-serve`). | fix/live-dcs-failures-triage | +| `FIX-LIVE-DCS-FAILURES` | ✅ pending merge | Triage the 10 (of 18) failures from the first live run (2026-07-10, `test-results-run2.xml`) not already fixed by `POST-FULLGAS-FIXES` — all 10 turned out to be cross-scenario state contamination, cleared by a mission reload (48/48 green, no src/test change). | fix/live-dcs-failures-triage | ### Planned lots diff --git a/CHANGELOG.md b/CHANGELOG.md index 886ff91..59c805c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -36,8 +36,10 @@ Versioning follows [Semantic Versioning](https://semver.org/). DCS-editor `beacon.ogg`-class bug (large embedded `l10n/DEFAULT/` resources). - First full live run (`--no-ai`, 45 scenarios) against a real DCS mission: 27/45 passed after a mission reload cleared cross-scenario state contamination between 4 vehicle/JTAC-family - scenarios (F-120/F-121/F-122/F-123); ~13 remaining failures are reproducible across repeated - fresh runs and need investigation (tracked informally, not yet ticketed). + scenarios (F-120/F-121/F-122/F-123); 18 failures total. 8 were fixed by the FullGas review + round above; the remaining 10 (`FIX-LIVE-DCS-FAILURES` lot) turned out to be the same class of + cross-scenario state contamination, not real bugs — a fresh mission reload cleared all of them + (48/48 `auto`/`auto-check` scenarios green, confirmed on two consecutive fresh runs). ### CI / tooling