CF-06: harden oracle execution cost and corpus qualification runtime

[TheHalfMoon]

Purpose

Track a performance/reliability hardening gap discovered while qualifying the HL7 comparator fix against the full frozen CF-10 corpus.

This issue does not change CF-06 semantics, the production HL7 pin, the frozen corpus, or PR #11. It is not authorization to parallelize or batch oracle work without preserving determinism and fail-closed behavior.

Observed execution shape

The current CF-10 runner invokes the HL7 Java adapter once per changed matched canonical StructureDefinition. Each invocation has a bounded 60-second timeout and process-tree termination on timeout, which is correct fail-closed behavior.

The qualification workflow hl7-upstream-fix-qualification runs the complete frozen corpus twice sequentially (A then B) under a 180-minute job timeout. Once the two known upstream comparator crashes are fixed, the run reaches substantially more oracle comparisons instead of failing early, making full A/B qualification materially more expensive.

Current qualification run:

run: 31918255417
job: 95093683595
step: Run frozen CF-10 corpus twice with combined candidate

Required invariants

Any optimization must preserve:

• CF-03 as semantic authority;
• HL7 as advisory oracle only;
• exact per-resource identity and evidence;
• bounded stdout/stderr;
• bounded execution time;
• fail-closed subprocess errors/timeouts;
• deterministic report ordering and byte output;
• no hidden network/package-cache dependency;
• unchanged frozen CF-10 case selection and package bytes;
• no silent skipping of failed resources.

Candidate directions to evaluate after CF-10 qualification

1. reuse a long-lived validated JVM/oracle worker with an explicit request/response protocol;
2. bounded deterministic worker-pool concurrency with canonical result reordering;
3. batch multiple StructureDefinition comparisons inside one adapter invocation while retaining per-resource failure identity;
4. cache immutable parsed package/context state within a single corpus run;
5. separate per-case or per-side workflow jobs only if retained evidence and A/B determinism remain straightforward to audit.

No direction is pre-authorized by this issue.

Acceptance

A hardening change should demonstrate, on the frozen corpus:

• equivalent normalized oracle evidence to the current one-process-per-resource baseline;
• byte-identical deterministic summaries;
• explicit regression coverage for timeout/process failure behavior;
• bounded concurrency/resource use;
• no weakening of CF-06 failure semantics;
• materially lower full-corpus runtime measured from retained CI evidence.

Scheduling

Do not modify the currently running qualification proof. Evaluate this only after run 31918255417 has completed and its evidence has been classified.

[TheHalfMoon]

Measured runtime evidence is now available from successful qualification run 31918255417 / job 95093683595.

job started:     2026-08-16T00:52:19Z
job completed:   2026-08-16T02:55:29Z
total job time:  2h 03m 10s

full frozen A/B step started:   2026-08-16T01:18:53Z
full frozen A/B step completed: 2026-08-16T02:55:25Z
A/B step time:                  1h 36m 32s

The expensive A/B step nevertheless completed successfully inside the current 180-minute job timeout, and final evidence enforcement passed. This gives #16 a concrete baseline rather than a theoretical performance concern.

The hardening target should therefore be measured against a 1h 36m 32s full frozen A/B baseline on this GitHub runner while preserving byte-identical summaries, per-resource failure identity, bounded subprocess behavior, and all existing CF-06 fail-closed invariants. No runtime optimization is being applied to the already-proven CF-10 qualification.

[TheHalfMoon]

Hardening sequence after successful CF-10 proof

Code inspection and the successful 31918255417 runtime baseline make the safest optimization order clearer.

H0 — workflow-only parallel A/B qualification (first)

The current experiment runs frozen corpus A and then frozen corpus B serially inside one job, then compares the two summaries. A and B are intentionally independent executions, so the first optimization should move them into separate bounded CI jobs that run concurrently and retain their evidence independently, followed by a small enforcement job that downloads both artifacts and requires:

status_a == 0
status_b == 0
cmp(summary_a, summary_b) == 0

This changes no commandF production code, no CF-06 semantics, no oracle source identity, no per-resource timeout behavior, and no frozen corpus inputs. Each A/B job must independently verify exact commandF/HL7/adapter identities and run with network isolation for the corpus execution.

Acceptance for H0:

• both sides use the exact same pinned inputs and adapter bytes;
• each side retains the complete case evidence and status independently;
• the comparison/enforcement job fails closed if either artifact/status is missing or non-zero;
• byte-identical deterministic summaries remain mandatory;
• measured wall-clock improvement is recorded against the 1h 36m 32s serial A/B step baseline.

H1 — bounded deterministic process concurrency (only if H0 is insufficient)

The current Rust path invokes run_hl7_oracle_adapter once per eligible canonical StructureDefinition. Every invocation preserves its own 60-second timeout, bounded stdout/stderr, and process-tree termination. If additional runtime reduction is still needed after H0, evaluate a small fixed worker bound around these existing invocations rather than changing the adapter protocol first.

Required H1 properties:

• fixed explicit concurrency bound, not unbounded available_parallelism() fan-out;
• canonical input order assigned before execution;
• results reordered to canonical input order before reconciliation/serialization;
• no new work scheduled after a failure is known;
• already-running workers are joined/terminated safely;
• reported failure is the earliest failing canonical input that the sequential baseline would encounter, not whichever worker finishes first;
• existing per-resource timeout/output/process-tree behavior remains unchanged;
• baseline and concurrent modes produce equivalent normalized per-resource oracle evidence and byte-identical final summaries.

Defer — long-lived JVM and batch protocol

Do not start with a long-lived JVM or multi-resource batch adapter. Main.compare() currently loads core, side, and context packages fresh for each comparison, so those designs may eventually yield the largest CPU/startup savings, but they also change important failure boundaries: per-resource timeout identity, subprocess isolation, partial-output handling, and which resource is attributable when a worker hangs or exits. Those should be considered only after H0/H1 measurements show they are necessary.

Current recommendation

H0 workflow-only parallel A/B
    -> measure retained CI runtime/evidence
    -> H1 bounded deterministic process concurrency only if needed
    -> batch/long-lived JVM deferred

No production mutation or CF-06 pin/source change is authorized by this sequencing note.

[TheHalfMoon]

H0 runtime experiment is now running on an isolated workflow-only branch. No production CF-06 code, oracle identity, frozen corpus, PR #11 content, or main branch state is changed.

Exact H0 identity:

branch: exp/cf06-oracle-runtime-h0-parallel-ab
base:   c95cc7665b0dce6b91d2dcff31791361215abe82
head:   edb57179eb719e9c0ef4edd2b1676eaeffd77e47
run:    32050152005

The diff from the proven experiment base is workflow-only. The H0 benchmark runs the two complete frozen-corpus executions concurrently with independent work roots, then requires all of the existing deterministic/evidence properties before accepting any runtime result:

• runner A/B success;
• each emitted summary equals its retained internal summary.json;
• A and B byte-identical;
• both summaries equal the already-proven serial summary SHA-256 074594e630c02f10c4bec9bcb903e153edaeb88a04c65636ff11c11301547409;
• both retained closure bindings verify;
• all three frozen cases remain complete;
• metadata-only repository boundary remains clean;
• retained per-side and wall-clock runtime evidence is uploaded.

Baseline for comparison remains the proven serial A/B step from run 31918255417: 5792 seconds (1h 36m 32s).

H0 deliberately does not introduce long-lived JVM state, batching, per-resource concurrency, altered timeout semantics, or altered failure identity. Those remain deferred until this lowest-risk workflow-only experiment is measured.

[TheHalfMoon]

H0 isolation preflight was checked against the exact experiment head before interpreting runtime results. corpus run creates all mutable acquisition/cache/lock/evidence state under the supplied work root, and resolve_state() creates PackageCache under work_root/states/<case>/<before|after>/cache. prepare_fresh_work_root() fails if the root already exists. H0 supplies distinct /tmp/cf10-h0-a and /tmp/cf10-h0-b roots, so A/B do not share commandF package cache, lockfiles, or evidence state. The shared oracle JAR and Java executable are read-only inputs. This makes concurrent A/B a valid workflow-level isolation experiment; any external registry/network contention remains part of runner wall-time behavior and cannot silently alter accepted package bytes because existing digest/closure/equality gates still apply.

[TheHalfMoon]

H0 acceptance is tightened before any result is classified as adopted.

A single GitHub-hosted-runner timing sample is not sufficient to accept runtime hardening because host CPU/network variance can materially affect wall time. The first successful H0 run should therefore be treated only as a single-run candidate proof.

For H0_ACCEPT_FOR_RUNTIME_HARDENING, require at least 3 independent successful runs on the exact same workflow/head (or an equivalent replicated benchmark set) with all semantic/evidence gates passing every time:

• runner A = 0 and runner B = 0;
• A/B internal summaries match retained summaries;
• A and B are byte-identical;
• both equal the proven serial summary SHA-256 074594e630c02f10c4bec9bcb903e153edaeb88a04c65636ff11c11301547409;
• closure binding passes for both sides;
• C001/C002/C003 remain complete;
• repository boundary remains clean;
• no change to CF-06 semantics, production pin, frozen corpus, or per-resource fail-closed behavior.

Record per-run A seconds, B seconds, concurrent wall seconds, runner/image identity when available, then compare median and worst-case wall time against the proven serial A/B baseline of 5792 seconds. A material improvement should be repeatable, not a one-off favorable runner.

The currently running benchmark 32050152005 remains untouched and is the first candidate sample only.

[TheHalfMoon]

CF-06 runtime hardening is now canonical and proven.

Canonical integration:

• PR perf(cf06): parallelize deterministic oracle validation #17 — MERGED
• canonical main at integration: 3df371773d9f8ba284a4ff5938e710aa63bccec8
• canonical tree: f0f009fe8abcab99b3a1d339d400b63af4a7b486
• candidate head: 098f644c15d894c109711f432cbaac720d138fb8
• candidate-to-merge tree equivalence: no changed files
• exact-head ci: SUCCESS
• exact-head cf06-oracle: SUCCESS
• CodeRabbit: SUCCESS / no actionable findings

Production topology now runs the independent oracle-self-smoke and oracle-changed-profile suites on separate runners, followed by fail-closed oracle-proof enforcement. Existing CF-06 assertions, failure-path coverage, deterministic behavior, frozen corpus boundaries, and the production HL7 identity (6.10.2, source d06577dbc5c62c74a2a8823fbc4830a3024d5b0b) remain unchanged.

Measured production-candidate wall-time improvement was approximately 13.6% versus the comparable canonical single-job baseline. The earlier H0 benchmark remains experimental provenance and is not a product performance guarantee.

Classification: CF06_RUNTIME_HARDENING_CLOSED_CANONICAL_PROVEN.

This closeout does not authorize a production HL7 pin change, CF-10 merge, semantic reinterpretation, or CF-12.