Skip to content

feat: Add a Python Callout implementation for the Prompt Amendment / Insertion - #356

Open
victorwoli wants to merge 7 commits into
GoogleCloudPlatform:mainfrom
victorwoli:prompt-amender-gateway
Open

feat: Add a Python Callout implementation for the Prompt Amendment / Insertion#356
victorwoli wants to merge 7 commits into
GoogleCloudPlatform:mainfrom
victorwoli:prompt-amender-gateway

Conversation

@victorwoli

@victorwoli victorwoli commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Prompt-Amender: ext_proc callout for centralized LLM prompt governance

Adds prompt_amender, an ext_proc callout that matches requests by caller identity/host/path against a hot-reloadable YAML ruleset and mutates systemInstruction in flight — injecting safety guardrails and context without client code changes or redeploys.

Highlights

  • Header-phase glob matching (spiffe://``/principalSet:// normalized); mode_override skips body buffering for unmatched traffic.

  • Body-phase mutation (prepend/append/replace/template, sandboxed Jinja2, compiled once); accepts both systemInstruction and system_instruction, inserting one if missing.

  • Correctly handles Envoy's streamed body delivery — non-final chunks withheld via clear_body rather than naively accumulated, so multi-chunk requests aren't corrupted; failures reconstruct the original body (fail-open) or reject (fail-closed).

  • Hot-reload from GCS/git with atomic swap and validation-reject.

  • Structured JSON logs (no raw prompt text) and real OTEL metrics/tracing.

Included: callout code, Terraform (with strip_client_spiffe_id defaulting true — identity headers only trustworthy behind a real gateway), Dockerfile, README, and 38 unit tests (no gRPC/network deps).

Follow-up: callouts/python/requirements-test.txt needs Jinja2==3.1.4 added (see requirements-test-addition.txt) — a shared file outside this PR's diff.

@victorwoli
victorwoli requested a review from a team as a code owner August 12, 2026 17:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant