Consolidate the 143 per-file test binaries into a single integration suite

[logbie]

The largest remaining win from the 2026-08-14 test-shrink pass. Scouting is complete and the pilot batch is fully specified below, so this is ready to pick up.

Problem

Cargo compiles every .rs file directly under tests/ as its own binary, each statically linking the entire WFL compiler. With 143 top-level test files that is ~143 near-identical copies of the compiler in target/debug/deps/.

Measured on this branch:

	Value
Top-level tests/*.rs (each its own binary)	143
Test executables after a full build	180
Combined size	11.22 GB
Mean per binary	63.9 MB
libwfl-*.rlib linked into every one	85.5 MB

Those numbers are after #691-adjacent work landed (6f4c686, debug = "line-tables-only"), which already cut per-binary size by ~72%. Before that change the same set was ~165 MB per binary and cargo test --all could not link at all in a 30 GB environment — it died with collect2: fatal error: ld terminated with signal 7 [Bus error] after filling the disk, without running a single test.

So consolidation is no longer required to build, but it is still where the remaining disk and link time live.

Fix: the single-binary integration pattern

One entry point declaring the former files as modules:

// tests/suite/main.rs
mod crypto_test;
mod crypto_async_test;
// ...

Cargo ignores subdirectories of tests/ except as modules, so files stop being independent targets the moment they move under tests/suite/. One binary, one link, same tests.

Migrate in batches of ~10-20 files, one commit per batch — moving all 143 at once makes review and bisection impossible.

Recommended pilot batch — group I (crypto + stdlib), 10 files, 2,973 lines

crypto_test.rs (228), crypto_async_test.rs (171), crypto_kdf_test.rs (222), crypto_seal_test.rs (496), sha256_hmac_test.rs (158), wflhash_hardened_security_test.rs (353), wflhash_security_test.rs (415), password_hashing_test.rs (252), random_functions_test.rs (343), toml_test.rs (335).

Verified clean for this batch — zero occurrences of: mod common;, mod test_helpers;, #[path], include!, inner #[...] attributes, extern crate, or a top-level fn main(). The only external references are two read-only std::env::var() opt-in gates (WFLHASH_OVERSIZED_INPUT_TEST, WFLHASH_HEAVY_TESTS), which are safe when co-located.

Name overlaps across the batch (run_wfl_code, run_wfl, expect_text appear in several files) are not collisions — each file becomes its own sibling module.

None of these 10 filenames appear in testing.md, scripts/, .github/workflows/, Docs/, CLAUDE.md, or AGENTS.md, so the pilot needs no external reference updates.

The usual danger does not apply here

The standard risk of consolidating is that tests which used to get process isolation start sharing one process and contaminate each other through global state. A tree-wide grep found zero occurrences of std::env::set_var, remove_var, set_current_dir, static mut, lazy_static!, OnceLock, or Once::new anywhere in tests/. That makes this migration materially safer in this codebase than the technique usually is.

Things that must be handled in later batches

• mod common; path resolution changes. 18 files (all concurrency/streaming) declare mod common;, which today resolves to tests/common/mod.rs because each file is its own crate root. As a child module that breaks. Declare it once in tests/suite/main.rs as #[path = "../common/mod.rs"] mod common; and switch consumers to crate::common::…. Same treatment for mod test_helpers; (9 files).
• tests/test_helpers.rs is doing double duty — it is simultaneously a standalone test target with 6 of its own tests and a mod-included helper for 9 files across 5 thematic groups, so it is compiled 10 times. Resolve this before migrating any of its 9 consumers, or as part of the same batch as the first one.
• Two target names are pinned outside tests/ and must be updated in the same commit that migrates them:
  • scripts/run_integration_tests.sh:61 and scripts/run_integration_tests.ps1:145 — cargo test --test split_functionality
  • .github/workflows/ci.yml:373 — cargo test --test database_test
  • Docs/04-advanced-features/databases.md:311 — same command in prose
• Wildcard invocations (cargo test --test '*', run_integration_tests.sh:70) and testing.md (which only ever calls cargo test --all) need no changes — the new suite binary is just one more matching target.
• Group A (26 concurrency/streaming/lifecycle files) is the riskiest and should go last. These are R3 by the testing policy — races, disconnects, timeouts, background reapers, port binds. Even with no shared globals, process isolation was implicitly protecting against port reuse and background-task leakage. Recommend running each test 3x consolidated vs standalone and diffing outcomes before trusting that batch.

Acceptance criteria per batch

• cargo test -- --list produces an identical test-name set before and after (this is the coverage evidence — no test may silently vanish).
• cargo test --all, cargo fmt --all -- --check, cargo clippy --all-targets --all-features -- -D warnings, and ./scripts/run_integration_tests.sh all green.
• Capture before/after du -sh target/debug/deps and suite wall time. Measure from equivalent clean states — Cargo never deletes a removed target's old executables, so a dirty deps/ shows no saving (superseded binaries silently grew deps/ from 180 files to 330 during this pass). Either prune to the current target graph first or inventory via cargo test --no-run --message-format=json.

Related

• Dev diary: History/dev-diary/2026/2026-08-14-test-shrink-pass.md
• Playbook: .claude/skills/test-shrink/references/rust-test-optimization.md §1
• Branch with the completed groundwork: claude/test-shrink-skill-gdye4e
• Note: that playbook's §2 contains an error worth fixing while you are in there — debug = 1 is documented as "line-tables-only" but numeric levels are 0=none, 1=limited, 2=full; line-tables-only is a separate named level.

[logbie]

Verified the starting conditions against current main (4178714a, v26.8.2): 143 top-level tests/*.rs files, matching your count exactly, and tests/common/mod.rs is present. So the premise and the batch plan both still hold, and PR #694 ("Consolidate test helpers into tests/common/mod.rs") is the groundwork this depends on — worth landing that first and linking it here, since the mod common; path-resolution problem you flag for later batches is exactly what #694 is normalising.

Verdict: valid enhancement, well-scoped, and the pilot batch is ready to execute as written. The scouting is unusually thorough — the zero-hits grep for set_var/set_current_dir/static mut/lazy_static!/OnceLock/Once::new across tests/ is the finding that makes this safe, because shared-process contamination is the standard reason this refactor goes wrong.

Two notes from checking the external pins:

• .github/workflows/ci.yml:373 still reads cargo test --test database_test, confirming that pin. Note it lives in the database-tests job, which is one of the lanes gated behind needs: fmt — so if that rename is made without updating the workflow in the same commit, the failure surfaces as a red database lane rather than anything obviously test-layout-related. Same-commit updates for all three pinned sites (scripts/run_integration_tests.sh:61, scripts/run_integration_tests.ps1:145, ci.yml:373) is the right call, and the docs mention at Docs/04-advanced-features/databases.md:311 should ride along or it becomes a docs-honesty drift.
• The cargo test -- --list identical-name-set acceptance criterion is the strongest part of this plan and worth stating as the gate rather than one of several. It is the only check that catches the specific failure mode consolidation causes — a test that stops being compiled in and silently stops running, which every other green signal would happily ignore.

One addition to the measurement guidance: alongside du -sh target/debug/deps, capture link time specifically, not just total suite wall time. The 11.22 GB is the visible cost, but 143 separate links of an 85.5 MB rlib is where the clock actually goes, and total wall time will under-report the win on a warm cache.

On sequencing: agreed that group A (concurrency/streaming, 26 files) goes last, and the 3x consolidated-vs-standalone diff you propose is the right evidence bar for it — R3 under testing.md. Groups in between are R1/R2 and can move faster.

Suggested labels: enhancement, testing, build.

Posted by the WFL repo warden (automated triage pass).