Test incremental_compilation_covers_all_models silently skips all TEST_MODELS (path-prefix mismatch)

[bpowers]

Summary

The test incremental_compilation_covers_all_models in src/simlin-engine/tests/simulate.rs silently skips every entry in TEST_MODELS. Its name and panic message both claim it verifies incremental compilation over the combined ALL_INCREMENTALLY_COMPILABLE_MODELS + TEST_MODELS corpus, but in practice it only exercises ALL_INCREMENTALLY_COMPILABLE_MODELS.

Root cause

The two model lists use different path conventions:

• ALL_INCREMENTALLY_COMPILABLE_MODELS entries carry a ../../ prefix, e.g. "../../test/sdeverywhere/models/sum/sum.xmile" (tests/simulate.rs:3112-3132).
• TEST_MODELS entries do not, e.g. "test/test-models/samples/arrays/a2a/a2a.stmx".

The test chains both lists and opens each path directly with a bare File::open:

for model_path in ALL_INCREMENTALLY_COMPILABLE_MODELS
    .iter()
    .chain(TEST_MODELS.iter())
{
    let f = match File::open(model_path) {
        Ok(f) => f,
        Err(_) => continue,   // <-- TEST_MODELS paths land here
    };
    ...
}

(tests/simulate.rs:3142-3149)

Engine tests run with the working directory set to the crate dir src/simlin-engine. A TEST_MODELS path opened without the prefix resolves to src/simlin-engine/test/..., which does not exist, so File::open returns Err, the Err(_) => continue arm fires, and the model is silently skipped.

Verified from the crate dir src/simlin-engine:

• test -e src/simlin-engine/test/test-models/samples/arrays/a2a/a2a.stmx -> MISSING (the bare path)
• test -e test/test-models/samples/arrays/a2a/a2a.stmx -> EXISTS (the ../../-prefixed path)

That the bare TEST_MODELS paths need a prefix is corroborated elsewhere in the same file: tests/simulate.rs:1752 opens TEST_MODELS[0] via an explicit File::open(format!("../../{}", TEST_MODELS[0])), and the generated corpus::* tests prepend the prefix via concat!("../../", $path).

Why it matters

• Misleading test: the test name (..._covers_all_models) and behavior diverge. A reader (or a future change that breaks incremental compilation specifically on a TEST_MODELS model) would trust a green result that never touched that corpus.
• Inflated panic denominator: the failure message computes the total as ALL_INCREMENTALLY_COMPILABLE_MODELS.len() + TEST_MODELS.len() (tests/simulate.rs:3175-3179), so a reported "N of M models failed" overstates M by TEST_MODELS.len() while the loop only ever fails on ALL_INCREMENTALLY_COMPILABLE_MODELS.
• Silent-skip fragility: the Err(_) => continue pattern hides both the path-convention bug and any future file-open regression. Even within ALL_INCREMENTALLY_COMPILABLE_MODELS, a renamed/removed fixture would be silently skipped rather than failing.

Severity / scope

Low. Actual incremental-compilation coverage of the TEST_MODELS corpus does exist elsewhere: the generated corpus::* tests call simulate_path -> compile_vm -> the incremental salsa path on each TEST_MODELS entry (with the ../../ prefix). So this is a misleading/redundant test, not a hole in real coverage.

Component(s) affected

• src/simlin-engine/tests/simulate.rs (the incremental_compilation_covers_all_models test; the TEST_MODELS / ALL_INCREMENTALLY_COMPILABLE_MODELS constants)

Possible approaches

1. Prefix the TEST_MODELS paths with ../../ when opening them in this test (matching the corpus::* generated tests' concat!("../../", $path) and the tests/simulate.rs:1752 usage), or normalize both lists onto a single prefix convention.
2. Replace the Err(_) => continue for the File::open step with a hard failure (push to failures or expect) so a missing/renamed fixture cannot be silently skipped. The downstream xmile::project_from_reader and the non-XMILE else { continue } arms can keep skipping as intended, but a file that is supposed to exist failing to open should be loud.
3. After fixing, confirm the loop actually iterates and compiles every TEST_MODELS entry (e.g. assert the number of models actually attempted equals the denominator, instead of trusting the static .len() sum).

Discovered

Identified while refactoring src/simlin-engine/tests/simulate.rs.