The v0.9 test audit proved three suite tests could pass while being wrong, each by breaking the thing it covers and watching the test stay green (c65324b). All three are fixed. Eleven more suspects were identified but not fixed, each with the concrete ROM change it would fail to notice. This issue tracks working through them.
This is the repo's canonical failure mode — "a check that can pass while being wrong" — and the fixed three show it is not hypothetical: battle_entry reported green in 2 seconds on a ROM with jsl Ot6SeedShields deleted, i.e. with every enemy in the game unbreakable, while its own header calls it "the quick iteration loop for battle/break-system changes."
Ranked, highest value first. Each line names the change that would slip through:
battle_hudclobber.lua:159-161 — asserts junkFrames == 0 with a control that proves only the hazard, never that the HUD renders. Make Ot6BgHudFlush_ext's veil permanent, or delete the under-enemy HUD entirely: still green. The "repaints once the tiles are whole" half — the thing the file is named for — is untested. battle_whelkwipe.lua:242 already contains the control this needs, so it is a copy, not a design problem.
battle_divines.lua:241-247 — the "honest reduced fallback" has no positive witness, and guard HP is re-pinned every drive tick (:122, pinGuardHp set :220), so damage is structurally unobservable. Make Ot6Oblivion's unbroken arm power-0 or bail entirely and all four assertions pass — a non-Broken Oblivion silently becomes a wasted turn plus 99 MP.
battle_gaufight.lua:587-596 — the shield ledger is asserted against a natural world-map encounter (walkIntoEncounter, :225-255), unpinned, and nothing asserts any hit body is bludgeon-weak. On a non-bludgeon roll the assertion degenerates to false == false. Delete the OT6_BLUDG arm from Ot6ClassChip and it is a coin flip whether the suite notices. This is also a flake waiting to be blamed on something else. One-line fix, or pin the formation.
battle_runic.lua:779-788 — driveUntil checks its predicate before the body, so if turns(celes) were already 0 the walk returns immediately and droppedEarly stays false. The entering value is logged (:776-777) but never asserted. One line.
battle_slotsboot.lua:315-326 — assert-in-one-branch; the gated arm has only an H.log. On any run where reel 1 lands the 7 in a $2f49.2-forbidden battle, the entire tier-3 promise silently vanishes. Nondeterministic, which is worse than stably wrong.
battle_reveal_poweron.lua:102-105 — accepts $BF or $FF or $00 for a '?' cell while running under OT6_RAM_POWERON=AllOnes, so an unpopulated shadow line reads $FF and passes. No fieldHudPresent() or glyphCanary() control anywhere in the file. Gut Ot6BgHudLine's weakness-cell write: green.
visual_f2.lua:31-40 — if alive then … with no assertion that alive was ever true; both lobos dying inside the blind 600-frame window skips the headline check.
battle_mpcost.lua:188-192 — self-detecting mode inverts every expectation. Turn OT6_MP_COSTS off in the shipped build and the default suite run detects "off", asserts SwdTech is free, and passes. Only Makefile:140's byte-compare, outside the file, stands in the way.
battle_class.lua:243-245 and battle_codex.lua:36-37 (slots 2 and 3 only) — true write-then-read tautologies. H.loadState writes 0x4f/0x38 into all four codex page headers on every load (lib/ot6.lua:306-312) and codexBase() always returns one of those four, so they cannot fail for any ROM change. battle_codex's slot-1 case is real — it deliberately plants 0x37 at :22.
battle_reveal.lua:95-97 (residual after the fix) — part 1's real anchor clsW == 0x02 is gated on sp == 0 with no assertion that a species-0 monster is present. If the battle_doorstep formation changes, part 1 collapses to three zero-checks that InitBattle's clear satisfies alone.
- Inert but harmless, listed so nobody re-finds them:
battle_breakvector.lua:167,225,300,325 compute purely from Lua literals; battle_breaktbl.lua:133,135 and battle_breakfloor.lua:61 are guaranteed by their own locator sequences; battle_bushido.lua:224-229 is an empty-bodied for loop that reads like a ceiling sweep.
The bar for closing any of these
Demonstrate the failure. Make the ROM change named above, watch the test stay green, fix it, watch it go red on the same ROM, restore, watch it pass. An argument that a test is sound is worth very little here — all three of the confirmed cases looked fine on inspection.
And do not weaken anything to get green. If an assertion turns out to be wrong rather than weak, say so explicitly.
The v0.9 test audit proved three suite tests could pass while being wrong, each by breaking the thing it covers and watching the test stay green (
c65324b). All three are fixed. Eleven more suspects were identified but not fixed, each with the concrete ROM change it would fail to notice. This issue tracks working through them.This is the repo's canonical failure mode — "a check that can pass while being wrong" — and the fixed three show it is not hypothetical:
battle_entryreported green in 2 seconds on a ROM withjsl Ot6SeedShieldsdeleted, i.e. with every enemy in the game unbreakable, while its own header calls it "the quick iteration loop for battle/break-system changes."Ranked, highest value first. Each line names the change that would slip through:
battle_hudclobber.lua:159-161— assertsjunkFrames == 0with a control that proves only the hazard, never that the HUD renders. MakeOt6BgHudFlush_ext's veil permanent, or delete the under-enemy HUD entirely: still green. The "repaints once the tiles are whole" half — the thing the file is named for — is untested.battle_whelkwipe.lua:242already contains the control this needs, so it is a copy, not a design problem.battle_divines.lua:241-247— the "honest reduced fallback" has no positive witness, and guard HP is re-pinned every drive tick (:122,pinGuardHpset:220), so damage is structurally unobservable. MakeOt6Oblivion's unbroken arm power-0 or bail entirely and all four assertions pass — a non-Broken Oblivion silently becomes a wasted turn plus 99 MP.battle_gaufight.lua:587-596— the shield ledger is asserted against a natural world-map encounter (walkIntoEncounter,:225-255), unpinned, and nothing asserts any hit body is bludgeon-weak. On a non-bludgeon roll the assertion degenerates tofalse == false. Delete theOT6_BLUDGarm fromOt6ClassChipand it is a coin flip whether the suite notices. This is also a flake waiting to be blamed on something else. One-line fix, or pin the formation.battle_runic.lua:779-788—driveUntilchecks its predicate before the body, so ifturns(celes)were already 0 the walk returns immediately anddroppedEarlystays false. The entering value is logged (:776-777) but never asserted. One line.battle_slotsboot.lua:315-326— assert-in-one-branch; thegatedarm has only anH.log. On any run where reel 1 lands the 7 in a$2f49.2-forbidden battle, the entire tier-3 promise silently vanishes. Nondeterministic, which is worse than stably wrong.battle_reveal_poweron.lua:102-105— accepts$BFor$FFor$00for a'?'cell while running underOT6_RAM_POWERON=AllOnes, so an unpopulated shadow line reads$FFand passes. NofieldHudPresent()orglyphCanary()control anywhere in the file. GutOt6BgHudLine's weakness-cell write: green.visual_f2.lua:31-40—if alive then …with no assertion thatalivewas ever true; both lobos dying inside the blind 600-frame window skips the headline check.battle_mpcost.lua:188-192— self-detectingmodeinverts every expectation. TurnOT6_MP_COSTSoff in the shipped build and the default suite run detects "off", asserts SwdTech is free, and passes. OnlyMakefile:140's byte-compare, outside the file, stands in the way.battle_class.lua:243-245andbattle_codex.lua:36-37(slots 2 and 3 only) — true write-then-read tautologies.H.loadStatewrites0x4f/0x38into all four codex page headers on every load (lib/ot6.lua:306-312) andcodexBase()always returns one of those four, so they cannot fail for any ROM change.battle_codex's slot-1 case is real — it deliberately plants0x37at:22.battle_reveal.lua:95-97(residual after the fix) — part 1's real anchorclsW == 0x02is gated onsp == 0with no assertion that a species-0 monster is present. If thebattle_doorstepformation changes, part 1 collapses to three zero-checks thatInitBattle's clear satisfies alone.battle_breakvector.lua:167,225,300,325compute purely from Lua literals;battle_breaktbl.lua:133,135andbattle_breakfloor.lua:61are guaranteed by their own locator sequences;battle_bushido.lua:224-229is an empty-bodiedforloop that reads like a ceiling sweep.The bar for closing any of these
Demonstrate the failure. Make the ROM change named above, watch the test stay green, fix it, watch it go red on the same ROM, restore, watch it pass. An argument that a test is sound is worth very little here — all three of the confirmed cases looked fine on inspection.
And do not weaken anything to get green. If an assertion turns out to be wrong rather than weak, say so explicitly.