diff --git a/.backlog/FIX-LIVE-DCS-FAILURES/PRD.md b/.backlog/FIX-LIVE-DCS-FAILURES/PRD.md new file mode 100644 index 0000000..9a049d1 --- /dev/null +++ b/.backlog/FIX-LIVE-DCS-FAILURES/PRD.md @@ -0,0 +1,38 @@ +# Lot FIX-LIVE-DCS-FAILURES — triage the first live-run failures + +Status: ✅ done — full 48/48 green live, no src/test changes needed (see ticket 01) +Branch: fix/live-dcs-failures-triage → PR → develop +Program: re-tooling CTLD on the VMCT model (see `.backlog/README.md`). Sibling to +`POST-FULLGAS-FIXES` (already merged, PR #22) and `CLEANUP-LEGACY-DCS-TESTS`. + +## Problem Statement + +The first full live run of `tools/integration-runner/run_scenarios.py --no-ai` against a real DCS +mission (2026-07-10, `test-results-run2.xml`, 45 scenarios) came back 27 passed / 18 failed. This +was mentioned in `CHANGELOG.md` ("~13 remaining failures ... tracked informally, not yet +ticketed") but never actually turned into backlog tickets. + +`POST-FULLGAS-FIXES` has since fixed 8 of the 18 as a side effect of FullGas's review answers. +This lot tracks the remaining ones and needs a fresh live run to confirm the current count before +triaging each. + +## Already fixed (verify only, no new work here) + +Covered by `POST-FULLGAS-FIXES` tickets 01–06 — re-confirm on the next live run, do not re-open: + +- `scenario_fq_vehicle_whole_transport.lua` (ticket 01) +- `U-108_modValidatorHeliportProbeOffMap.lua`, `U-108_modValidatorHeliportWarnAndSkip.lua` (tickets 02/03) +- `F-117_reconDisabledScanShowsExplicitMessageNotSilent.lua`, `F-118_reconToggleOFFMarksRemovedImmediatelyEvenWhenNoLay.lua` (ticket 04) +- `scenario_fr_ai_zones.lua` (ticket 05) — already re-checked standalone post-fix: PASS (`test-results.xml`, 2026-07-12) +- `scenario_farp_countryside_spawn.lua`, `scenario_farp_metal_spawn.lua` (ticket 06) + +## Tickets + +| # | Scope | Nature | +|---|-------|--------| +| 01 | Triage the 10 remaining failures from `test-results-run2.xml`, re-verify live, fix or re-ticket each | live-DCS + src/test | + +## Non-goals + +- The 8 already-fixed scenarios above (verification is folded into ticket 01's fresh run, not separate work). +- Porting/purging the 194 FullGas relics — `CLEANUP-LEGACY-DCS-TESTS`. diff --git a/.backlog/FIX-LIVE-DCS-FAILURES/tickets/01-triage-remaining-failures.md b/.backlog/FIX-LIVE-DCS-FAILURES/tickets/01-triage-remaining-failures.md new file mode 100644 index 0000000..214dd26 --- /dev/null +++ b/.backlog/FIX-LIVE-DCS-FAILURES/tickets/01-triage-remaining-failures.md @@ -0,0 +1,58 @@ +# 01 — Triage the 10 remaining live-run failures + +Status: ✅ done +Type: hybrid (live DCS re-run + src/test fix per finding) + +## Resolution (2026-07-12) + +Fresh live run (`--no-ai`, current `develop` + POST-FULLGAS-FIXES already merged): **48/48 +`auto`/`auto-check` scenarios PASS**, including all 10 below — confirmed stable across two +consecutive runs. No `src`/test change was needed. + +Root cause: the same class of issue already documented for F-120/F-121/F-122/F-123 in the +2026-07-10 run — cross-scenario state contamination accumulated over a long-running mission +session (shared globals: `MenuManager` singleton, scheduler, `U-108` probe run-once guards). +A fresh mission (reload) starts with clean state and every scenario resolves correctly, +including `scenario_mt05_crate_vehicle.lua`'s apparent `isExist()` crash and `F-119`'s radius +mismatch — neither reproduces from a clean state, so not real bugs in +`CTLDCrateManager:getLoadedCrateWeight` or the recon icon geometry. + +**Takeaway for future live runs**: reload the mission (`Shift+R`) before a full `--no-ai` sweep, +not just after `src/` changes — long-running sessions accumulate cross-scenario state that +produces false failures. + +## Context + +Source: `test-results-run2.xml` (2026-07-10 live run, 45 scenarios, 18 failures). 8 of the 18 are +already covered by `POST-FULLGAS-FIXES` (see PRD "Already fixed"). The 10 below are untouched. + +## What to do + +1. Re-run live (`python tools/integration-runner/run_scenarios.py --no-ai --inject-ctld`) against + current `develop` to confirm which of the 10 still fail (some may have moved since 07-10). +2. For each confirmed failure: read `tests/dcs/CTLD.log` for the ones that only say "see + CTLD.log", root-cause, and either fix `src/` or rewrite the stale scenario/assertion — + whichever is wrong. Follow legacy parity (`migration/source/CTLD.lua`) as the reference. +3. Re-run to confirm PASS before closing. + +## The 10 failures to triage + +| Scenario | Failure (2026-07-10) | First read | +|---|---|---| +| `F-119_reconAAIconCircleToAllFilled2ApexLineToAllNoBare3L.lua` | Circle radius: `scale=1.0 got=67.5 exp=15.75`, `scale=2.0 got=135 exp=31.5` (both ratios ≈4.29×) | Consistent ratio across both cases — looks like a real unit/formula bug, not a stale test | +| `scenario_ai_transport.lua` | F-134.1/.2/.3/.5: `embarkFromTroopZone`/`disembarkAll` mocks never called | Wiring/mock likely stale vs current `CTLD_core.lua` AI-transport flow | +| `scenario_b3_load_crate_from_menu.lua` | FAIL 3/8 — see `CTLD.log` | — | +| `scenario_cl9_pickup_zones.lua` | FAIL 2/5 — see `CTLD.log` | — | +| `scenario_crate_menu_flight_visibility.lua` | FAIL 11/19 — see `CTLD.log` | — | +| `scenario_feature_p_caps_rename.lua` | FAIL 1/44 — see `CTLD.log` | — | +| `scenario_mt05_crate_vehicle.lua` | Runtime crash: `CTLD.lua:12304: attempt to call method 'isExist' (a nil value)` | `CTLDCrateManager:getLoadedCrateWeight` (`src/CTLD_crate.lua:1052`) calls `crate.loadedBy:isExist()` without a nil/type guard — likely a real nil-safety bug, prioritize (crash, not just assertion) | +| `U-051_oRDERSortingDCSRebuildFollowsOrderFieldNotInsertio.lua` | Menu order assertions (Troops/Crates/FOB by `order` field) fail | — | +| `U-052_pagination13Items9NextPageAtF10.lua` | Pagination assertions (9 items page 1, "→ Next Page") fail | — | +| `U-053_enabledFlagDisabledNodeAbsentFromDCSRebuild.lua` | Enabled-flag assertions (Troops/Beacons rendered) fail | — | + +## Acceptance criteria + +- [x] Fresh live run confirms the current failure set (may differ from 2026-07-10). +- [x] Each of the 10 is either fixed (src or test) with a green re-run, or explicitly deferred with + a reason recorded here. +- [x] `tests/recette.md` / `CHANGELOG.md` updated to drop the "not yet ticketed" note. diff --git a/.backlog/POST-FULLGAS-FIXES/tickets/01-fq-regression.md b/.backlog/POST-FULLGAS-FIXES/tickets/01-fq-regression.md index 435dbb6..b323ace 100644 --- a/.backlog/POST-FULLGAS-FIXES/tickets/01-fq-regression.md +++ b/.backlog/POST-FULLGAS-FIXES/tickets/01-fq-regression.md @@ -9,3 +9,7 @@ silently disabling whole-vehicle spawn from Request Equipment. Restored the logi b1ddfe4 reference (both objects exist in our repo, read-only). src + rebuild CTLD.lua. Validated live: scenario_fq_vehicle_whole_transport PASS 9/9 (was 1/9). + +L4 pilotPassive confirmed live (2026-07-12, David, UH-1H): whole-vehicle option visible and +spawns correctly from F10 → CTLD → Request Equipment — closes the L4 gap this menu change had +skipped before merge. diff --git a/.backlog/README.md b/.backlog/README.md index cecf357..2397321 100644 --- a/.backlog/README.md +++ b/.backlog/README.md @@ -14,7 +14,7 @@ authored **per lot, when the lot is started** (not in batch). | Lot | Status | Description | Branch | |-----|--------|-------------|--------| -| `REINTEGRATE-ORPHAN-TESTS` | ✅ pending merge | Rebuilt coverage for the 46 features tested only by dead FullGas relics — 6 busted tickets (158 tests, green in CI) + 1 live-DCS ticket (3 scenarios: BCN 30/30, RCN 10/10, VEH 22/22). Precedes & unblocks `CLEANUP-LEGACY-DCS-TESTS`. 10 relics were false orphans (already covered); 6 deferred to FullGas (`dev/fullgas-report.md`). PR #20. | feature/reintegrate-orphan-tests | +| `FIX-LIVE-DCS-FAILURES` | ✅ pending merge | Triage the 10 (of 18) failures from the first live run (2026-07-10, `test-results-run2.xml`) not already fixed by `POST-FULLGAS-FIXES` — all 10 turned out to be cross-scenario state contamination, cleared by a mission reload (48/48 green, no src/test change). | fix/live-dcs-failures-triage | ### Planned lots @@ -39,6 +39,8 @@ authored **per lot, when the lot is started** (not in batch). | `DCS-BRIDGE-MCP` ✅ | Migrate DCS integration testing from Witchcraft to VEAF-dcs-bridge: `.mcp.json` + project-local venv, scenario return contract, 79 scenarios + 4 templates migrated, `integration-testing` skill, Witchcraft fully retired (PR #14). Uncovered ~194 dead FullGas relics → `CLEANUP-LEGACY-DCS-TESTS`. | | `INTEGRATION-TEST-TAGS` ✅ | `-- @tier: auto\|auto-check\|ia` header on the 79 scenarios + 4 templates (43/2/34 split); fixed F-122's unresolved verdict found during the classification audit (PR #15). | | `INTEGRATION-TEST-RUNNER` ✅ | Dependency-free `tools/integration-runner/run_scenarios.py`: discovers + tier-filters scenarios, drives them over dcs-serve REST, polls async ones, writes a JUnit report; `--no-ai` mode runs every `auto`/`auto-check` scenario headlessly. 31 stdlib unit tests. Closes the DCS-bridge triptych. | +| `REINTEGRATE-ORPHAN-TESTS` ✅ | Rebuilt coverage for the 46 features tested only by dead FullGas relics — 6 busted tickets (158 tests, green in CI) + 1 live-DCS ticket (3 scenarios: BCN 30/30, RCN 10/10, VEH 22/22). 10 relics were false orphans (already covered); 6 deferred to FullGas (`dev/fullgas-report.md`). PR #20. | +| `POST-FULLGAS-FIXES` ✅ | Applied FullGas's review answers: Feature Q whole-vehicle spawn regression, AI-zone stock validation rewrite, U-108 heliport probe fixes, F-117/F-118 recon fixes, `coord_farp-1` mission static, README doc touch-ups. First full live run against real DCS (27/45) surfaced 18 failures, 8 fixed here as a side effect — remaining 10 → `FIX-LIVE-DCS-FAILURES`. PR #22. | ## Dropped lots diff --git a/CHANGELOG.md b/CHANGELOG.md index 886ff91..59c805c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -36,8 +36,10 @@ Versioning follows [Semantic Versioning](https://semver.org/). DCS-editor `beacon.ogg`-class bug (large embedded `l10n/DEFAULT/` resources). - First full live run (`--no-ai`, 45 scenarios) against a real DCS mission: 27/45 passed after a mission reload cleared cross-scenario state contamination between 4 vehicle/JTAC-family - scenarios (F-120/F-121/F-122/F-123); ~13 remaining failures are reproducible across repeated - fresh runs and need investigation (tracked informally, not yet ticketed). + scenarios (F-120/F-121/F-122/F-123); 18 failures total. 8 were fixed by the FullGas review + round above; the remaining 10 (`FIX-LIVE-DCS-FAILURES` lot) turned out to be the same class of + cross-scenario state contamination, not real bugs — a fresh mission reload cleared all of them + (48/48 `auto`/`auto-check` scenarios green, confirmed on two consecutive fresh runs). ### CI / tooling