feat: add agent red-teaming + guardrails cookbook - #12
Open
u7k4rs6 wants to merge 1 commit into
Open
Conversation
Adds agent_red_teaming/: a runnable example that fires a battery of adversarial inputs at a simple support agent, screens each turn at input and output with the FutureAGI guardrail pipeline (jailbreak, code-injection, secrets), and traces the run in FutureAGI Observe. Attack set mirrors the adversarial simulation personas: prompt injection, jailbreak, secret exfiltration, PII extraction, plus a benign control. Input scan blocks injection/jailbreak; output scan catches any credentials or PII the agent might leak under social engineering. Also adds the entry to the top-level README.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds an
agent_red_teamingcookbook: a runnable example that fires adversarial inputs at a simple support agent, screens each turn at input and output with the FutureAGI guardrail pipeline, and traces the run in Observe. Also adds the entry to the top-level README.Files added
Attack battery
prompt_injectionjailbreaksecret_exfiltrationpii_extractionbenign_controlThe two-layer model (input block + output leakage scan) reflects real-world deployment: social-engineering requests carry no regex signature at input, so output scanning is the second line of defense.
Why
No existing cookbook covers red-teaming or the guardrail pipeline. This gives users a copy-paste pattern for input and output guardrailing around an agent, with the attack set mapped to the adversarial simulation personas added in future-agi/future-agi#1092. Guardrail scanners run locally; only the agent call needs an LLM key.
Verification
Pipeline detection confirmed against the attack battery before commit: