Integrated walkthrough, regression matrix, and CI gate - #6
Merged
Conversation
Add the fixed walkthrough runner: it resets both services and proves every required case — vulnerable disclosure, secure projection, vulnerable mass assignment, secure whole-request rejection, the legitimate employee edit, the legitimate reviewer decision, and the non-owner control in both variants — printing the actor, both authorization verdicts, the HTTP outcome, the authorized key set, and the state diff for each, and exiting nonzero the moment one is missing. It accepts only its enumerated modes and knows only the Compose service names, so no argument can point it elsewhere. So it can assert audit-event cardinality without reading a log file, the demonstration boundary now exposes the bounded, already-redacted audit history, and reset clears it. Counting a delta across a full ring would silently read zero, so cases reset first and assert an absolute count. tests/test_regression_matrix.py maps every FR-015 row to the named test that proves it and fails if one is renamed or deleted. GitHub Actions runs the same `docker compose run --rm verify` the README documents, with no separate pipeline. The README now explains BOLA versus BOPLA, both wrong data flows, both halves of the fix, why client-side filtering and obscure property names are not authorization, and how the suite proves security alongside preserved legitimate behavior. The image builds with `uv sync --locked` so a lockfile that drifts from pyproject fails the build instead of installing something else. Refs #5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Turns the two verified services into a finished local teaching artifact: one deterministic
walkthrough, an explicit regression matrix, educational documentation, and a CI job that runs the
same gate the README documents.
The walkthrough
Real run against the containers:
Exit codes, observed:
full)0--mode secure(5/5 cases)0--mode nonsense2--target http://example.invalid2It only knows the two Compose service names, and there is no argument that redirects it or names a
different object. That it detects failure rather than always passing is proved in the suite: point
it at a service that cannot leak and the vulnerable cases fail; point the secure cases at the leaking
service and those fail.
Audit-event cardinality, and a trap avoided
The runner needs to assert "exactly one audit event" without reading a log file, so the demonstration
boundary now exposes the bounded, already-redacted audit history at
/demo/events, and/demo/resetclears it.
The first version counted a delta across that ring and passed in isolation while failing in the full
suite: the ring holds only rejection events, so once saturated the count is pinned at its maximum and
every delta reads zero — an assertion that could never fail. Cases now reset first and assert an
absolute count, and
test_the_audit_history_stays_boundedpins the bound itself.Regression matrix
tests/test_regression_matrix.pymaps eachFR-015row to the named test that proves it and assertsthat test exists, so renaming or deleting a load-bearing test fails the gate instead of quietly
shrinking the matrix. A second test pins the required row set so a row cannot disappear either.
Verification
Well under the five-minute budget. The image now builds with
uv sync --locked, so a lockfile thatdrifts from
pyproject.tomlfails the build rather than silently installing something else — whichis how the stale lock from moving
httpxinto the runtime dependencies was caught.Every documented manual command in the README was run literally against the running container: both
reads, the mixed-body rejection, the legitimate employee edit, the reviewer decision, the non-owner
control,
/demo/state,/demo/events, and/demo/reset.Documentation
The README now opens with the one command that shows everything, then explains BOLA versus BOPLA in a
table, both wrong data flows in a diagram, both halves of the fix (explicit response schema per
actor; explicit request schema plus assignment by name), and a short section on what property
authorization is not — client-side filtering, undocumented fields, hard-to-guess names, hidden form
fields, blocklists. It ends with the OWASP and CWE references. Every example uses only the fixed
fictional fixture, and the vulnerable service is marked intentionally unsafe and local-only
throughout.
Acceptance
tests/test_walkthrough.py400with one audit event and unchanged statetest_the_runner_fails_when_the_vulnerable_cases_cannot_hold,..._secure_cases_...test_the_runner_rejects_anything_but_its_enumerated_modes, exit codes aboveFR-015row maps to a named test, checked mechanicallytests/test_regression_matrix.pydocker compose run --rm verifystays the single gate, green from a clean checkout, well under five minutes.github/workflows/verify.ymlREADME.mdREADME.mdNotes
verifyagainst both variants over real loopback HTTP,which is what keeps the single gate hermetic. The containerized
walkthroughservice is the samecode, run against the two Compose services for a human to watch.
walkthroughuses anentrypointrather than acommand, sorun --rm walkthrough --mode secureappends the flag instead of replacing the invocation.
/demo/eventsexposes only events that were already redacted before reaching the log; a testasserts it carries no property name, value, or credential.
Deliberately out of scope
Licensing, publication metadata, the exposure review, and any visibility change (
SLICE-004).Closes #5