feat(security): PII/PHI redaction for telemetry + EU AI Act Article 15 export (#398)#442
feat(security): PII/PHI redaction for telemetry + EU AI Act Article 15 export (#398)#442Prateeks16 wants to merge 2 commits into
Conversation
…reerevanth#398) CMP-003/004 — auto-redact PII/PHI before telemetry is persisted, and surface an EU AI Act Article 15 conformity export. - agentwatch/security/redaction.py: a Redactor that masks PII/PHI as [REDACTED]. Uses Microsoft Presidio when installed (new optional `redaction` extra), otherwise falls back to the GDPR (CMP-001) and HIPAA (CMP-003) regex detectors — reusing their patterns, no hard dependency. Provides redact()/redact_payload()/redact_tool_call() helpers. - core/watcher.py: opt-in `GenericAdapter(redact=True)` scrubs tool-call payloads (raw_command + arguments) before events are published/persisted. - api/server.py: GET /api/v1/governance/eu-ai-act-report returns the Article 15 technical documentation + conformity assessment as JSON. - Tests: PII/PHI masking, payload recursion, tool-call redaction, the watcher opt-in, and the EU AI Act endpoint. - Docs: CHANGELOG, MASTERLIST (CMP-003/004), and the `redaction` extra.
|
Warning Review limit reached
More reviews will be available in 11 minutes and 47 seconds. Learn how PR review limits work. Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file). ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits. 🚦 How do rate limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (9)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
🧪 PR Test Results
Python 3.12 · commit a6a2e30 |
There was a problem hiding this comment.
Pull request overview
Adds an opt-in telemetry redaction layer to scrub PII/PHI before persistence and introduces an EU AI Act Article 15 JSON export endpoint under the existing governance module.
Changes:
- Introduce
agentwatch.security.redactionwith Presidio (optional) + regex fallback redaction helpers for text, nested payloads, and tool calls. - Wire redaction into
GenericAdapterviaredact=True, and addGET /api/v1/governance/eu-ai-act-report. - Add a
redactionoptional dependency extra plus tests and status/changelog updates.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
agentwatch/security/redaction.py |
New redaction module implementing Presidio/regex PII/PHI masking utilities. |
agentwatch/core/watcher.py |
Adds opt-in redaction hook for tool-call telemetry in the watcher adapter. |
agentwatch/api/server.py |
Adds EU AI Act Article 15 report export endpoint. |
pyproject.toml |
Adds optional redaction extra for Presidio dependencies. |
tests/test_redaction.py |
Adds unit/integration tests for redaction behavior, watcher integration, and the new endpoint. |
MASTERLIST_STATUS.md |
Updates compliance phase entries to reflect added deliverables/tests. |
CHANGELOG.md |
Documents the new redaction feature and report endpoint under Unreleased. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # ── Safety gate (async path — full check_event with approval) ── | ||
| if is_tool_like: | ||
| tool_call = _build_tool_call_data(method_name, args, kwargs) | ||
| tool_call = self._maybe_redact(_build_tool_call_data(method_name, args, kwargs)) | ||
| safety_event = AgentEvent( |
| # ── Safety gate (sync path — pattern match only, no approval) ── | ||
| if is_tool_like: | ||
| tool_call = _build_tool_call_data(method_name, args, kwargs) | ||
| tool_call = self._maybe_redact(_build_tool_call_data(method_name, args, kwargs)) | ||
| try: | ||
| blocked, reasons = self._safety_engine.check_tool_call_sync(tool_call) |
| """EU AI Act Article 15 conformity export (CMP-004). | ||
|
|
||
| Maps AgentWatch's safety telemetry to the Article 15 requirements and | ||
| returns the technical documentation plus a conformity assessment as JSON. | ||
| """ | ||
| from agentwatch.governance.eu_ai_act import EUAIActPackage, TechnicalDocumentation | ||
|
|
||
| doc = TechnicalDocumentation( | ||
| system_name="AgentWatch-monitored AI system", | ||
| intended_purpose="Observability, safety, and reliability layer for AI agents", | ||
| risk_category="high", | ||
| data_governance={"pii_phi_redaction": "enabled", "retention": "policy-driven"}, | ||
| robustness_evidence=[ |
| from agentwatch.governance.gdpr import _PII_PATTERNS | ||
| from agentwatch.governance.hipaa import _PHI_PATTERNS | ||
|
|
|
@Prateeks16 The redaction functionality is useful and the overall direction makes sense. However, there are a few issues that need to be addressed before merge:
Once these items are addressed, I'll take another look. |
…export, public patterns Maintainer feedback on sreerevanth#398: 1. Safety must evaluate the raw tool call, not a redacted copy. Redaction ran before the safety check, masking signals (paths/secrets/identifiers) the SafetyEngine relies on and potentially altering block/allow decisions. The watcher now checks the raw payload and scrubs PII/PHI only when building the event that is published/persisted (new _redact_event helper). Added a regression test asserting safety sees the raw command while the published event is redacted. 2. The EU AI Act Article 15 export reported static literals. It now derives data governance, accuracy metrics, robustness evidence, human-oversight description, and record-keeping (decision log) from live safety telemetry and the active policy. 3. Stop importing private governance internals. gdpr/hipaa now expose public pii_patterns() / phi_patterns() accessors; the redactor uses those.
|
Thanks for the review — all three addressed in the latest push:
Full suite green except one pre-existing CLI test ( |
Closes #398
Summary
Implements the two CMP-003/CMP-004 deliverables from the issue: auto-redaction of PII/PHI in telemetry before it is persisted, and an EU AI Act Article 15 conformity export endpoint.
The compliance/governance layer already exists under
agentwatch/governance/(gdpr.py,hipaa.py,eu_ai_act.py,compliance_reporter.py+ the/governance/compliance-reportendpoint). The missing piece — and the focus of this PR — is the redaction layer and wiring it into the telemetry path.What's added
agentwatch/security/redaction.py— aRedactorthat masks PII and PHI as[REDACTED]:presidio-analyzer/presidio-anonymizer) for NER-based detection when installed (new optionalredactionextra), and otherwise falls back to the regex detectors already maintained by the GDPR (CMP-001) and HIPAA (CMP-003) engines — so there's a single source of truth for the patterns and no hard dependency on the heavyweight Presidio stack.redact(),redact_payload()(recursive),redact_tool_call().core/watcher.py— opt-inGenericAdapter(redact=True)scrubs each tool call'sraw_command+argumentsbefore the event is published and persisted.api/server.py—GET /api/v1/governance/eu-ai-act-reportreturns the Article 15 technical documentation + conformity assessment as JSON, mapping AgentWatch's safety telemetry (risk scoring, red-team harness, blast-radius) to the Article 15 requirements.Acceptance criteria
[REDACTED]before persistence — covered byredaction.py+ the watcher opt-in./governance/compliance-reportplus the new/governance/eu-ai-act-reportArticle 15 export.Design notes
redact=True.pip install agentwatch[redaction]).Testing
tests/test_redaction.py: PII masking (SSN/email/phone), PHI masking (MRN/diagnosis/ICD), recursive payload redaction,redact_tool_call, the watcher opt-in (on/off), and the EU AI Act endpoint viaTestClient.Note on scope / process
CONTRIBUTING flags security/compliance features for Discord discussion first — flagging here; happy to adjust. The issue's suggested paths (
agentwatch/compliance/eu_ai_act.py) map to the existingagentwatch/governance/module, which this PR builds on rather than duplicating.The repo-wide
ruff check .gate has pre-existing failures in unrelated test files onmain; this PR's changed files are lint-clean.