Some paths cannot be proven against a fake. Image decoding at real sizes on real
hardware, a genuine TLS handshake against a genuine certificate chain, a real
server's behaviour under a real sign-in. Pretending those are covered by the
headless suite is worse than admitting they are not.
So they get a separate harness with an honest name, run deliberately, and nothing
about it is allowed to make the headless suite in #20 conditional.
What the harness is
A separate target, not part of the default test command, named so that its name
says what it needs. It refuses to run rather than skipping silently when what it
needs is absent, and it prints what was missing.
Its results are never counted as part of the headless suite's coverage, and no
gate treats a skipped integration run as a pass.
Blocked on #20, which is where the headless suite this harness must not make
conditional is built. The label on this issue said so and the body did not, which
is the half a reader checks.
Done when
The harness exists as a separate target, running the default test command never
invokes it, invoking it without its prerequisites fails with a message naming
what was missing, and CONTRIBUTING.md says what it covers that the headless
suite cannot.
Some paths cannot be proven against a fake. Image decoding at real sizes on real
hardware, a genuine TLS handshake against a genuine certificate chain, a real
server's behaviour under a real sign-in. Pretending those are covered by the
headless suite is worse than admitting they are not.
So they get a separate harness with an honest name, run deliberately, and nothing
about it is allowed to make the headless suite in #20 conditional.
What the harness is
A separate target, not part of the default test command, named so that its name
says what it needs. It refuses to run rather than skipping silently when what it
needs is absent, and it prints what was missing.
Its results are never counted as part of the headless suite's coverage, and no
gate treats a skipped integration run as a pass.
Blocked on #20, which is where the headless suite this harness must not make
conditional is built. The label on this issue said so and the body did not, which
is the half a reader checks.
Done when
The harness exists as a separate target, running the default test command never
invokes it, invoking it without its prerequisites fails with a message naming
what was missing, and
CONTRIBUTING.mdsays what it covers that the headlesssuite cannot.