feat(ai): run external guardrails on the non-streaming response#556
Merged
Conversation
Extends the external HTTP guardrail providers from the input path to the output path. `post_call` / `during_call` entries in `guardrails.external` now inspect the materialized non-streaming response before it is cached or sent, blocking with the same 403 guardrail_violation envelope as the built-in output guardrails. The output block is now computed from the sync pipeline OR the external providers, so an origin with only external output guardrails (no compiled output pipeline) is covered too. The run_input/run_output runners share one driver parameterized by phase. Streaming-response external guardrails and AWS Bedrock (SigV4) remain follow-ups. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X19S6eQzKKExZ9RUPAHuGy
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Completes the external HTTP guardrail providers (shipped input-side in #551) on the output path.
post_call/during_callentries inguardrails.externalnow inspect the materialized non-streaming response before it is cached or sent, blocking with the same403 guardrail_violationenvelope the built-in output guardrails use.How
run_input_external_guardrailsand a newrun_output_external_guardrailsnow share one privaterun_external_guardrails(cfgs, content, on_input)driver, parameterized by phase (is_input()vsis_output()).relay_ai_response_with_cache, the output-guardrail block is now computed from the sync pipeline or the external providers, so an origin configured with only external output guardrails (no compiled output pipeline) is covered too. The external check runs only when the sync pipeline didn't already block, and only on 2xx response text. The existing block-handling (waste accounting, 403, cache-skip) is unchanged.Tests
run_outputskips an input-only (pre_call) guardrail without any HTTP call, and evaluates apost_callone.sbproxy-ai(973) andsbproxy-core(460) lib suites pass; clippy-D warningsand rustdoc-D warnings -D missing_docsclean. No config-schema change (theguardrails.externalfield already exists).Scope
Non-streaming output only. Streaming-response external guardrails (per-chunk async is expensive) and AWS Bedrock (SigV4
ApplyGuardrail) remain follow-ups. As with the input side, CI can compile and unit-test the runner/decision logic but cannot validate against a live external guardrail endpoint.