Skip to content

fix(conformance): keep the default adapter probe envelope-neutral - #1159

Open
doublewhy wants to merge 1 commit into
devfrom
197-run-314-envelope-neutral-probe
Open

fix(conformance): keep the default adapter probe envelope-neutral#1159
doublewhy wants to merge 1 commit into
devfrom
197-run-314-envelope-neutral-probe

Conversation

@doublewhy

Copy link
Copy Markdown

Plain-language summary

  • Context: RUN-314 (Define the reference-backend contract and conformance evidence (RUN-314) #197) requires the released conformance harness to be invocable by an independently installed backend without private knowledge; the test(runtime): characterize backend result integrity #1150 falsification matrix noted run_target_conformance with default arguments fails even the honest reference backend.
  • Problem: Since authored OS identity started carrying through realization (feat: carry authored OS identity through realization #1141), the default probe scenario's os: linux demands exact SEM-218 os-family corroboration that no hermetic in-process backend declares — so target-provisioning and target-snapshot silently fail for every honest backend (reference and stub alike), and nothing pinned them.
  • Fix: Drop the authored OS from the default probe scenario (mirroring the canonical cross-backend-minimal decision that leaving os unset keeps a scenario admissible on more than one envelope) and pin both adapter probes passing for every hermetic backend.

Issue mapping

Summary

  • raes_conformance/conformance/target_probes.py: the default conformance scenario no longer authors os: linux; the comment records why (an authored OS is an exact realization requirement, not adapter mechanics — the SEM-218 probes own that concern).
  • tests/test_reference_backend_conformance.py: new test_adapter_probes_pass_for_every_hermetic_backend pins target-provisioning and target-snapshot passing for the reference and stub targets, so the probe cannot silently regress again.
  • The pinned fail-closed posture is untouched: report.passed stays False for the reference target (opacity and constructive-envelope probes still report unsupported), exactly as test_reference_target_requires_opacity_probe_for_positive_declaration requires.

Compatibility

  • No SDL field, schema, portable contract, or CLI changes. The default probe scenario is a harness input, not a published contract; callers who supplied their own reference_scenario are unaffected.

Verification

  • Reference and stub targets both pass target-provisioning and target-snapshot; full report posture unchanged.
  • nox -s tests: 7,012 passed; coverage gate green.
  • Ruff format and lint clean; tools/check_repo_policy.py pass.

🤖 Generated with Claude Code

Carrying authored OS identity through realization (#1141) turned the
default target-conformance probe scenario's 'os: linux' into an exact
SEM-218 os-family requirement demanding presence corroboration that no
hermetic in-process backend declares. Since then target-provisioning
and target-snapshot silently failed for every honest backend (reference
and stub alike), with no test pinning them.

Drop the authored OS from the default probe scenario — mirroring the
canonical cross-backend-minimal decision that leaving it unset is what
keeps a scenario admissible on more than one envelope — and pin both
adapter probes passing for every hermetic backend so this cannot drift
silently again. The issue #663 runner parameter remains the escape
hatch for bounded backends, and the pinned fail-closed posture of the
opacity and constructive-envelope probes (report.passed is False) is
unchanged.

Advances RUN-314's acceptance criterion that the released conformance
harness be invocable by a downstream backend with default arguments.

Verification: reference and stub targets both pass target-provisioning
and target-snapshot; nox -s tests green (7,012 tests); ruff clean;
check_repo_policy pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant