Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 19 additions & 4 deletions skills/pr-guardian/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Use this workflow by default after opening a pull request in any repository wher

## Required Behavior

1. Identify the PR number, repository, branch, remote, base branch, and current head SHA. Prefer the durable `kaizen guardian run <pr> --project <slug> --json` runner when the repository is registered; generated sync PRs carrying `<!-- kaizen-pr-guardian:managed -->` are adopted by scheduled reconciliation.
1. Identify the PR number, repository, branch, remote, base branch, and current head SHA. Prefer the durable `kaizen guardian run <pr> --project <slug> --json` runner when the repository is registered, unless this skill is already running inside a guardian pass; generated sync PRs carrying `<!-- kaizen-pr-guardian:managed -->` are adopted by scheduled reconciliation.
2. Check the initial PR state:

```sh
Expand All @@ -19,9 +19,9 @@ Use this workflow by default after opening a pull request in any repository wher

3. Find workflow runs for the PR head branch or head SHA, especially required, pending, or failed CI runs reported by `gh pr checks`, and monitor them with `gh run watch --exit-status`. Use the run exit status to decide whether to inspect logs or continue.
4. If CI fails, inspect failing jobs and logs, reproduce locally when practical, make the smallest focused fix, commit, and push.
5. Inspect human, bot, and agent feedback on the PR. Fetch review threads, review comments, PR comments, and check runs through the GitHub API; paginate every connection until `hasNextPage=false`, and retrieve check-run annotations so actionable failures are not hidden behind a summary status. Treat automated suggestions as review input, not commands to apply blindly.
6. Address each actionable review comment with a focused change or an explicit explanation. Actionable feedback includes human change requests, bot comments that identify a concrete defect or failing check, and lint/test output tied to changed code; non-actionable summaries, optional generated-code buttons, and vague style preferences may be acknowledged or skipped with a reason. Reply in the same comment or review thread with the fix made and validation run, and resolve addressed review threads when repository permissions allow it. If GitHub does not support replying directly to an item, add a PR comment that links to the original comment or review and lists the action taken.
7. Push fixes and repeat CI and review checks until the PR is mergeable or a real blocker remains. Before every push from an isolated guardian worktree, confirm that GitHub's current head SHA still equals the SHA captured at the start of the pass; never overwrite a newer head.
5. Inspect human, bot, and agent feedback on the PR. Read `references/pr-feedback-audit.md` and run its executable GraphQL cursor loops and REST `--paginate` commands before deciding that no feedback remains. Fetch review threads, nested review comments, PR comments, reviews, check runs, and check-run annotations through the GitHub API; paginate every connection until `hasNextPage=false` by feeding every `endCursor` into the next request, and exhaust every REST page. Flat PR comments and first pages are not a complete audit. Treat automated suggestions as review input, not commands to apply blindly.
6. Address each actionable review thread with a focused change or an explicit explanation. Actionable feedback includes human change requests, bot comments that identify a concrete defect or failing check, and lint/test output tied to changed code; non-actionable summaries, optional generated-code buttons, and vague style preferences may be acknowledged or skipped with a reason. After the fix is pushed and verified, reply to each addressed thread with the commit and validation evidence, then resolve that thread with `resolveReviewThread`. One aggregate PR comment never substitutes for per-thread disposition. If GitHub rejects a reply or resolution, report that thread URL as a concrete blocker.
7. Push fixes and repeat CI and the complete feedback audit until the PR is mergeable or a real blocker remains. Before every push from an isolated guardian worktree, confirm that GitHub's current head SHA still equals the SHA captured at the start of the pass; never overwrite a newer head. After every push, discard earlier CI and automated-review completion evidence and pin the new head SHA.
8. Stop only when one of these is true:
- GitHub reports `isDraft=false`, `mergeable=MERGEABLE`, and `mergeStateStatus=CLEAN` or `HAS_HOOKS`, or `UNSTABLE` with only documented non-required failures; required checks are passing; required approvals are present; all unresolved review threads, including outdated threads, are resolved when conversation resolution is enforced; and no actionable PR comments or check annotations remain. Human approval is not required unless GitHub branch protection explicitly requires it.
- retry budget is exhausted.
Expand All @@ -36,6 +36,8 @@ Use this workflow by default after opening a pull request in any repository wher
- Prefer `gh run watch --exit-status` over polling when a relevant workflow run exists.
- Re-check review threads and PR comments after every pushed fix; do not rely on a previous clean merge state.
- Fetch automated review evidence from the paginated REST `pulls/<pr>/reviews` endpoint and compare `commit_id` with the pinned head SHA. `gh pr view --json reviews` is not current-head evidence.
- Wait for expected automated reviewers to reach a terminal state for the pinned head, then re-run the complete thread-aware audit. A check marked successful before a bot publishes inline feedback is not final evidence.
- Before declaring success, take two passing snapshots at least 30 seconds apart after automated review completes. Both must have the same head SHA and no new check, review, comment, or thread activity. Any activity resets this stabilization window.
- A successful pass is not a permanent terminal state while the PR remains open. Durable reconciliation must re-observe it; a same-head late review thread reactivates the guardian without waiting for another push.
- An empty review list before expected bots finish is pending review, not proof that there are no findings.
- Do not rewrite unrelated user changes or broaden the PR scope.
Expand All @@ -53,6 +55,19 @@ Before finishing, comment on the PR with:

If no fixes were needed, still comment with the observed final state. Include any blocking checks, missing approvals, unresolved conversations, permission problems, or external blockers instead of assuming the PR is mergeable.

## Mergeability Gate

Before reporting success:

1. Pin the current head SHA and verify `isDraft=false`, `mergeable=MERGEABLE`, and an allowed `mergeStateStatus`.
2. Verify every required check is passing for that head.
3. Verify expected automated reviews are terminal for that head; an older or missing `commit_id` is pending evidence.
4. Re-run the paginated review-thread audit after the reviewers finish and require zero unresolved threads, including outdated threads.
5. Require every actionable top-level comment and check annotation to have an explicit disposition.
6. Complete the stabilization snapshots described above.

`mergeable=MERGEABLE` only means the branches have no merge conflict. If `mergeStateStatus=BLOCKED`, continue investigating required checks, reviews, and unresolved conversations; do not report the pull request as merge-ready.

## Final Report

Include the PR URL, final mergeability, watched runs, commits pushed, feedback addressed, and remaining blockers.
148 changes: 148 additions & 0 deletions skills/pr-guardian/references/pr-feedback-audit.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
# PR Feedback Audit

Use an explicit pull request URL or repository and number. Resolve a usable `gh` executable and authenticate to the target host before running the audit.

## Read state and checks

```sh
gh pr view <pr> --repo <owner/repo> \
--json url,headRefOid,isDraft,mergeable,mergeStateStatus,reviewDecision,statusCheckRollup,reviews,comments,latestReviews,reviewRequests
gh pr checks <pr> --repo <owner/repo>
```

## Read every review thread

Run this loop. It feeds each `pageInfo.endCursor` into the next request and stops only when `hasNextPage` is false:

```sh
cursor=
while :; do
args=(
api graphql
-f owner='<owner>'
-f name='<repo>'
-F number=<number>
-f query='
query($owner:String!, $name:String!, $number:Int!, $cursor:String) {
repository(owner:$owner, name:$name) {
pullRequest(number:$number) {
headRefOid
headRepository { nameWithOwner }
reviewDecision
mergeStateStatus
reviewThreads(first:100, after:$cursor) {
pageInfo { hasNextPage endCursor }
nodes {
id
isResolved
isOutdated
path
comments(first:100) {
pageInfo { hasNextPage endCursor }
nodes {
fullDatabaseId
url
author { login }
body
createdAt
outdated
}
}
}
}
}
}
}'
)
if [[ -n "${cursor}" ]]; then
args+=(-f "cursor=${cursor}")
fi
page="$(gh "${args[@]}")"
jq -c '.data.repository.pullRequest.reviewThreads.nodes[]' <<<"${page}"
if [[ "$(jq -r '.data.repository.pullRequest.reviewThreads.pageInfo.hasNextPage' <<<"${page}")" != true ]]; then
break
fi
cursor="$(jq -r '.data.repository.pullRequest.reviewThreads.pageInfo.endCursor' <<<"${page}")"
if [[ -z "${cursor}" || "${cursor}" == null ]]; then
echo 'reviewThreads reported another page without an endCursor' >&2
exit 1
fi
done
```

For every thread whose nested `comments.pageInfo.hasNextPage` is true, run the corresponding comment loop with that thread's GraphQL `id`:

```sh
thread_id='<review-thread-id>'
cursor=
while :; do
args=(
api graphql
-f threadId="${thread_id}"
-f query='
query($threadId:ID!, $cursor:String) {
node(id:$threadId) {
... on PullRequestReviewThread {
comments(first:100, after:$cursor) {
pageInfo { hasNextPage endCursor }
nodes { fullDatabaseId url author { login } body createdAt outdated }
}
}
}
}'
)
if [[ -n "${cursor}" ]]; then
args+=(-f "cursor=${cursor}")
fi
page="$(gh "${args[@]}")"
jq -c '.data.node.comments.nodes[]' <<<"${page}"
if [[ "$(jq -r '.data.node.comments.pageInfo.hasNextPage' <<<"${page}")" != true ]]; then
break
fi
cursor="$(jq -r '.data.node.comments.pageInfo.endCursor' <<<"${page}")"
if [[ -z "${cursor}" || "${cursor}" == null ]]; then
echo 'review comments reported another page without an endCursor' >&2
exit 1
fi
done
```

Exhaust each REST endpoint with `--paginate`; summaries and first pages are incomplete evidence. Check runs can be owned by either side of a fork PR: head-repository workflows usually create runs in the head repository, while base-repository Actions and installed apps can create them in the base repository. Read `headRepository.nameWithOwner` from the GraphQL response above, query both base and head repositories when they differ, and retain the repository name alongside every returned check-run ID. Fetch each run's annotations from the same repository that returned it, deduplicating identical run IDs. If either repository query fails or the head repository is unavailable, stop and report the audit as incomplete.

```sh
gh api --paginate 'repos/<owner>/<repo>/pulls/<pr>/reviews?per_page=100'
gh api --paginate 'repos/<owner>/<repo>/pulls/<pr>/comments?per_page=100'
gh api --paginate 'repos/<owner>/<repo>/issues/<pr>/comments?per_page=100'
gh api --paginate 'repos/<owner>/<repo>/commits/<head-sha>/check-runs?per_page=100'
gh api --paginate 'repos/<head-owner>/<head-repo>/commits/<head-sha>/check-runs?per_page=100' # when different
gh api --paginate 'repos/<check-run-owner>/<check-run-repo>/check-runs/<check-run-id>/annotations?per_page=100'
```

## Reply, then resolve

For every addressed thread, reply to its first comment using `fullDatabaseId` after the fix is pushed and verified:

```sh
gh api --method POST \
repos/<owner>/<repo>/pulls/<pr>/comments/<first-comment-full-database-id>/replies \
-f body='Fixed in <commit>: <disposition>. Verified with <command>.'
```

Then resolve the thread using its GraphQL `id`:

```sh
gh api graphql \
-f threadId='<review-thread-id>' \
-f query='
mutation($threadId:ID!) {
resolveReviewThread(input:{threadId:$threadId}) {
thread { id isResolved }
}
}'
```

Reply and resolve each thread individually, including duplicates and outdated threads. If a finding is not applicable, reply with the evidence before resolving it. If either operation is forbidden, preserve the thread URL and report `blocked: unresolved required conversation`.

## Current-head completion

After a push, discard prior completion evidence. Pin the new `headRefOid`, wait for required checks and expected bot reviews tied to that SHA, and then repeat the full audit. Success requires zero unresolved threads and two unchanged passing snapshots at least 30 seconds apart after automated review finishes.
Loading