Skip to content

fix(checkup): stop clear-text logging of orchestrator return in example (#1576)#1577

Open
DianaTao wants to merge 2 commits into
mainfrom
fix/checkup-example-clear-text-logging
Open

fix(checkup): stop clear-text logging of orchestrator return in example (#1576)#1577
DianaTao wants to merge 2 commits into
mainfrom
fix/checkup-example-clear-text-logging

Conversation

@DianaTao

Copy link
Copy Markdown
Collaborator

Fixes #1576.

Problem

context/agentic_checkup_orchestrator_example.py printed the full run_agentic_checkup_orchestrator return tuple in clear text (print(f"Message: {message}"), print(f"Model Used: {model}"), …). message can echo scrubbed-but-verbose command / LLM output, so CodeQL's "Clear-text logging of sensitive information" flags it. The example is regenerated by pdd example / checkup auto-heal, so the finding kept reappearing on PRs — it was auto-committed onto #1575 by chore: auto-heal prompt/example drift, surfacing 4 CodeQL alerts there.

Fix (durable)

  1. Prompt (pdd/prompts/agentic_checkup_orchestrator_python.prompt) — added instruction 8: usage examples / caller logging MUST NOT print the raw return (especially message/model) in clear text; demonstrate the outcome with a non-sensitive summary derived from success. This makes future pdd example regenerations stay clean instead of re-introducing the finding.
  2. Examplesuccess, *_rest = run_agentic_checkup_orchestrator(...) then print("Checkup orchestrator finished:", "success" if success else "did not pass"). No return field is logged in clear text.

Verification

🤖 Generated with Claude Code

…xample (#1576)

`context/agentic_checkup_orchestrator_example.py` printed the full
`run_agentic_checkup_orchestrator` return — `print(f"Message: {message}")`,
`print(f"Model Used: {model}")`, etc. `message` can echo scrubbed-but-verbose
command/LLM output, so CodeQL's "clear-text logging of sensitive information"
query flags it. The example is regenerated by `pdd example` / checkup auto-heal,
so the finding kept reappearing on PRs (auto-committed onto #1575).

- Prompt (`agentic_checkup_orchestrator_python.prompt`): add instruction 8 — usage
  examples / caller logging MUST NOT print the raw return tuple (especially
  `message`/`model`) in clear text; show a non-sensitive summary derived from
  `success`. Keeps future regenerations clean.
- Example: `success, *_rest = run_agentic_checkup_orchestrator(...)` then print a
  status string; no return field is logged in clear text.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@DianaTao

Copy link
Copy Markdown
Collaborator Author

Detailed bug analysis & fix

The bug

context/agentic_checkup_orchestrator_example.py printed the entire return tuple of run_agentic_checkup_orchestrator(...) in clear text:

success, message, cost, model = run_agentic_checkup_orchestrator(...)
print(f"Success: {success}")
print(f"Message: {message}")     # <-- flagged
print(f"Total Cost: ${cost:.4f}")
print(f"Model Used: {model}")    # <-- flagged

CodeQL's py/clear-text-logging-sensitive-data ("Clear-text logging of sensitive information") query flags these prints. The taint source is the orchestrator's return: message is built from command/agent output and, in the real (non-mocked) code path, can echo pytest output, command stderr, and CI logs — content that may embed ghp_/ghs_ tokens, Bearer headers, AWS keys, or https://user:token@host URLs. CodeQL traces the real function's return (not the example's mock), tags the tuple as sensitive, and reports every print of an unpacked field as a clear-text logging sink.

Why it kept coming back

This file is generated.pdd/meta/agentic_checkup_orchestrator_python.json records "command": "example", i.e. it's produced by pdd example from pdd/prompts/agentic_checkup_orchestrator_python.prompt. So checkup auto-heal regenerates it on prompt/example drift. That's exactly what happened on #1575: the bot pushed chore: auto-heal prompt/example drift for agentic_checkup_orchestrator, which regenerated the example with the same clear-text print block and surfaced 4 CodeQL alerts on that PR. Hand-fixing the file alone wouldn't stick — the next regeneration would re-introduce it.

Root cause

The generating prompt has extensive _scrub_secrets/_redact_secret requirements for the orchestrator's own console output (Step 5 failure visibility, base-ref fetch errors, etc.), but no convention governing how example/caller code may log the orchestrator's return value. So pdd example was free to print the full tuple.

The fix (durable, two parts)

  1. Prompt (pdd/prompts/agentic_checkup_orchestrator_python.prompt) — added instruction 8: usage examples / caller-side logging MUST NOT print the raw return tuple (especially message/model) in clear text; demonstrate the outcome with a non-sensitive summary derived from the boolean success. This makes every future regeneration safe, not just this snapshot.
  2. Example (context/agentic_checkup_orchestrator_example.py) — replaced the clear-text dump with:
    success, *_rest = run_agentic_checkup_orchestrator(...)
    # `message`/`model` in `_rest` can echo scrubbed-but-verbose command/LLM
    # output, so they are intentionally NOT logged in clear text here.
    print("Checkup orchestrator finished:", "success" if success else "did not pass")
    No return field is logged in clear text; success is consumed in a conditional that prints a literal, which keeps CodeQL clean.

Verification

Relationship to #1575

#1575 is the separate Step 7 gate fix (honor blocking: false / scope: out-of-scope in targeted PR mode). The CodeQL alerts that appeared on #1575 came from the auto-heal drift commit, which was reverted there. This PR (#1577) is the root-cause fix so the example never re-introduces the finding across future checkups.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

checkup example logs the orchestrator return in clear text (CodeQL: clear-text logging of sensitive information)

1 participant