Skip to content

fix: undifferentiate blocked_reason and re-screen replayed history (#255, #256) - #314

Merged
ckrough merged 1 commit into
mainfrom
issue/255
Jul 22, 2026
Merged

fix: undifferentiate blocked_reason and re-screen replayed history (#255, #256)#314
ckrough merged 1 commit into
mainfrom
issue/255

Conversation

@ckrough

@ckrough ckrough commented Jul 22, 2026

Copy link
Copy Markdown
Member

Summary

Two retriever RAG safety findings from audit #226, both surgical edits to RAGService.ask(). Hand-batched into one PR because both touch the same ask() method and cannot run as parallel workstreams.

#255blocked_reason disclosed which safety rail fired

A blocked /ask response set blocked_reason to the specific SafetyViolationType (prompt_injection, moderation_flagged, or hallucination), giving a caller a 3-way oracle: an attacker probing for bypasses could tell from the response alone which rail a payload tripped and tune against it.

Every block path now returns one undifferentiated BLOCKED_REASON ("blocked"). The specific violation type stays in server-side logs only, so observability is unchanged.

#256 — replayed conversation history was never re-screened

Only the newest question passed through check_input(). Prior conversation_history turns were copied and appended verbatim to every later model call, recirculating anything that entered a turn unscreened (for example via the retrieved-chunk path).

ask() now re-screens stored history via a new _screen_history() helper before generation, dropping and logging unsafe turns. This mirrors the existing retrieved-chunk screening idiom (drop + log, not block). Placed after the cache lookup so a cache hit skips the extra safety calls.

Tests

  • Oracle-closure test: two different rails (input injection, output hallucination) return an identical blocked_reason, proving no differentiation.
  • History drop test: an unsafe stored turn is removed and never reaches complete_with_history.
  • History keep test: all-safe history is replayed unchanged.
  • Existing block-reason assertions and the injection integration test updated to the collapsed contract.

Gate: ruff format/ruff check clean, mypy --strict clean, 385 unit tests pass at 89.5% coverage.

Closes #255
Closes #256

…eplayed history (#255, #256)

Two retriever RAG safety findings from audit #226, both in RAGService.ask().

#255: a blocked response set blocked_reason to the specific
SafetyViolationType (prompt_injection, moderation_flagged, or
hallucination), giving a caller a 3-way oracle to tell which rail a
payload tripped and tune bypasses against it. Every block path now
returns one undifferentiated BLOCKED_REASON ("blocked"); the specific
violation type stays in server-side logs only.

#256: only the newest question passed through check_input; prior
conversation_history turns were appended verbatim to every later model
call with no re-screening, recirculating anything that entered a turn
unscreened (for example via the retrieved-chunk path). ask() now
re-screens stored history before generation via _screen_history,
dropping and logging unsafe turns, mirroring the existing
retrieved-chunk screening. Placed after the cache lookup so a cache hit
skips the work.

Tests: oracle-closure test (two rails, identical reason) and history
drop/keep tests added; existing block-reason assertions and the
injection integration test updated to the collapsed contract.
@ckrough
ckrough merged commit 5633afd into main Jul 22, 2026
17 checks passed
@ckrough
ckrough deleted the issue/255 branch July 22, 2026 11:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant