docs: reframe WAF effectiveness as reported observations, not pass/fail gates#228
Merged
Conversation
…il gates Remove the absolute TPR >95% / FPR <10% detection targets from README.testing.md and course-project-report.md. Those numbers are properties of the CRS ruleset and paranoia level, not of Guard Proxy itself — enforcing them as CI gates would couple the project to something it doesn't own. README.testing.md: - Replace "Detection Targets (draft)" block with two explicit sections: "WAF effectiveness — reported, not gated" (pinned-baseline measurement with CRS SHA, PL, and corpus recorded per run) and "What CI actually enforces" (functional block/allow assertions + <20% overhead gate that the project actually owns). - Key thesis finding reframed as the before/after FP delta from applying a rule override, not an absolute FP rate. docs/course-project-report.md: - §1 (goals): add "Cele dodatkowe" block distinguishing primary owned goals from secondary reported observations (fidelity, overhead, tuning-delta); add explicit statement that TPR/FPR are CRS properties measured against a pinned baseline. - §12 (tests): split into "Bramy zaliczenia" (CI-enforced gates) vs "Pomiary raportowane" (TP/FP/delta — measured, not gated); note that every reported measurement must record its CRS reference point. - §13 (new): "Plan ewaluacji" — five-row table (E1–E5) with measurement, method, and type (gate vs reported); "punkt odniesienia" block specifying what must be recorded with every reported run. - §14 (formerly §13): Screenshots (renumbered). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Detection Targets (draft)block fromREADME.testing.md— theTPR >95%andFPR <10%thresholds were framed as CI gates but those numbers are properties of the CRS ruleset and paranoia level, not of Guard Proxy itself.docs/course-project-report.mdto match: new "Cele dodatkowe" block in §1, split test taxonomy in §12, and a new §13 "Plan ewaluacji" table (E1–E5, gate vs reported).Why
Enforcing absolute TPR/FPR thresholds as pass/fail gates would couple CI to CRS rule quality — something Guard Proxy doesn't control. The right framing is: Guard Proxy owns the wiring and management correctness; CRS owns detection quality. The thesis evaluation should report effectiveness measurements against a pinned baseline, not gate builds on them.
The key thesis finding is now framed as the before/after FP delta from applying a rule override on a fixed corpus — a reproducible, owned result that demonstrates the tuning workflow works.
What CI still enforces (unchanged behaviour)
Related
thesis/evaluation-plan.md§7 also updated (file is gitignored — lives outside the repo).Test plan
<10%/>95%thresholds (confirmed by grep — none found)