From 818d233e50869f5483bc3af30b2b3042c0002658 Mon Sep 17 00:00:00 2001 From: Andreas Ronneseth Date: Tue, 19 May 2026 15:12:34 -0700 Subject: [PATCH] refactor(workflows): overhaul AI reviewer prompt to reduce noise MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Check existing review threads before posting; never re-raise issues a human already responded to - Three clear options: silent approve, inline comments only, or defer on complex PRs - No summary comments on approval — just approve silently - Skip very large diffs (>1500 lines) entirely - Never post duplicate comments Co-authored-by: Cursor --- .github/workflows/claude-review.yaml | 56 +++++++++++++++++----------- 1 file changed, 35 insertions(+), 21 deletions(-) diff --git a/.github/workflows/claude-review.yaml b/.github/workflows/claude-review.yaml index fd21ee6..0df9321 100644 --- a/.github/workflows/claude-review.yaml +++ b/.github/workflows/claude-review.yaml @@ -17,6 +17,11 @@ on: required: false type: number default: 20 + max_diff_lines: + description: "Max diff lines before deferring to human review" + required: false + type: number + default: 1500 secrets: anthropic_api_key: required: true @@ -46,41 +51,50 @@ jobs: REPO: ${{ github.repository }} PR NUMBER: ${{ github.event.pull_request.number }} - ## Instructions - 1. Run `gh pr diff ${{ github.event.pull_request.number }}` to see the full diff. - 2. Analyze the changes against the criteria below. - 3. You MUST submit a formal review using one of these commands: - - APPROVE: `gh pr review ${{ github.event.pull_request.number }} --approve --body "AI Review: "` - - COMMENT: `gh pr review ${{ github.event.pull_request.number }} --comment --body "AI Review: "` - 4. If you have specific line-level feedback, post inline comments BEFORE submitting the review. + ## Step 1: Check scope and prior reviews + 1. Run `gh pr diff ${{ github.event.pull_request.number }}` to see the diff. + 2. If the diff exceeds ${{ inputs.max_diff_lines }} lines changed, or the PR is too complex to confidently review (many files, complex logic across multiple systems, architectural changes), defer to human review: + `gh pr review ${{ github.event.pull_request.number }} --comment --body "AI Review: Deferring to human review — this PR exceeds the automated review threshold."` + Then stop. Do NOT approve. Do NOT post inline comments. + 3. Run `gh api repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }}/comments` to read ALL existing inline review comments and reply threads. Also run `gh pr view ${{ github.event.pull_request.number }} --comments` for top-level PR comments. + 4. Any issue that was already raised AND responded to by a human is RESOLVED. Do not re-raise it, even if you still disagree. The human has the final call. + 5. Only look for NEW issues that have not been previously discussed. + + ## Step 2: Decide what to do + You have exactly two options. Pick ONE: + + **Option A — Approve:** + If the change is low-risk AND you have zero new concerns, approve: + `gh pr review ${{ github.event.pull_request.number }} --approve --body "AI Review: Approved"` - ## What to Review + **Option B — Post inline comments:** + If you found new concerns not already discussed, post them as inline comments on the relevant lines. Then submit a comment review: + `gh pr review ${{ github.event.pull_request.number }} --comment --body "AI Review: Found new concerns — see inline comments."` + Do NOT approve. + + ## What to look for - Correctness and logic errors - Security vulnerabilities (SQL injection, XSS, auth bypass, data exposure) - Data integrity risks (wrong column, missing validation, silent data loss) - Race conditions or concurrency issues - Breaking changes to public APIs or interfaces - ## What NOT to Review + ## What to ignore - Code style or formatting (linters handle this) - Missing comments or documentation - Naming preferences - Test coverage quantity - ## Decision: APPROVE or COMMENT - Submit an APPROVE review ONLY when ALL of these are true: - 1. Zero security concerns - 2. Zero correctness bugs - 3. Zero data integrity risks - 4. The change is low-risk: config change, dependency bump, small feature, test-only, docs, refactor with no behavioral change, template/view-only change - 5. You have zero inline comments to leave - - If ANY concern exists, submit a COMMENT review (never REQUEST_CHANGES). Post your findings as inline comments on the relevant lines. Let the human team make the final call. - - You MUST end by running one of the `gh pr review` commands above. Do not just write text — you must use the tool. + ## Rules + - NEVER use REQUEST_CHANGES. + - NEVER approve when you have concerns. Use Option B instead. + - NEVER re-raise an issue a human already responded to. + - NEVER post duplicate comments that are already on the PR. + - You MUST end by submitting a formal review via `gh pr review`. Do not just write text. + ## Repo-specific rules ${{ inputs.review_rules }} claude_args: >- --max-turns ${{ inputs.max_turns }} --model ${{ inputs.model }} - --allowedTools "mcp__github_inline_comment__create_inline_comment,Bash(gh pr review:*),Bash(gh pr comment:*),Bash(gh pr diff:*),Bash(gh pr view:*)" + --allowedTools "mcp__github_inline_comment__create_inline_comment,Bash(gh api:*),Bash(gh pr review:*),Bash(gh pr comment:*),Bash(gh pr diff:*),Bash(gh pr view:*)"