Skip to content

Fold "Close Stale PRs" into a deterministic pr-triage stale sweep#928

Merged
JanKrivanek merged 3 commits into
mainfrom
jankrivanek-merge-close-stale-prs-into-triage
Jul 23, 2026
Merged

Fold "Close Stale PRs" into a deterministic pr-triage stale sweep#928
JanKrivanek merged 3 commits into
mainfrom
jankrivanek-merge-close-stale-prs-into-triage

Conversation

@JanKrivanek

Copy link
Copy Markdown
Member

What & why

Fixes #915.

The agentic close-stale-prs.agent.md was unreliable at actually closing stale PRs and spent tokens on work that is fully deterministic. This PR extracts and preserves that behavior as a deterministic stale-PR sweep folded into the existing pr-triage workflow family, and deletes the agentic workflow.

Changes

  • New .github/scripts/pr-stale-sweep.sh — deterministic port of the stale policy (no model calls, no tokens):
    • Considers every open PR, including drafts.
    • "Last activity" = most recent non-bot comment/review; falls back to created_at. updated_at and all [bot] activity are ignored so the bot's own warning never resets the timer.
    • created ≤ 30 days → skip (too new)
    • 30 < inactivity ≤ 37 days → post a stale warning (once; guarded by a <!-- pr-triage:stale-warning --> marker)
    • inactivity > 37 days → close with a closing comment
    • Exempt: no-stale label; authors dotnet-maestro[bot] / dotnet-maestro
    • DRY_RUN and STALE_MAX (default 25) supported
  • .github/workflows/pr-triage-batch.yml — hosts the new stale-sweep job:
    • Weekly cron 17 4 * * 1 (plus the existing hourly triage cron)
    • workflow_dispatch inputs stale_sweep / stale_max for manual runs
    • Job-scoped pull-requests: write + issues: write permissions
  • Deleted close-stale-prs.agent.md and its compiled close-stale-prs.agent.lock.yml
  • Updated docs/design/pr-triage-workflows.md (new "Stale-PR sweep" section, diagram, no-stale note)

Why fold into pr-triage-batch.yml (not a new workflow)

pr-triage-batch.yml is the scheduled orchestrator and already exists on main with workflow_dispatch, so the sweep can be dispatched against this feature branch before merge for validation. A brand-new workflow file cannot be dispatched until it lands on the default branch.

Validation

  • actionlint -shellcheck= -pyflakes= passes on pr-triage-batch.yml.
  • bash -n passes on pr-stale-sweep.sh.

A live dry-run/real run against this repo and a sample closed PR will be linked in a follow-up comment.

Note

The stale_sweep / stale_max dispatch inputs are permanent (they're how the sweep is manually run), and they also serve as the trigger used to validate this PR from the branch.

Replace the agentic close-stale-prs.agent.md with a deterministic
stale-PR sweep hosted in pr-triage-batch.yml. The new stale-sweep job
runs .github/scripts/pr-stale-sweep.sh weekly (cron 17 4 * * 1) and on
manual dispatch (stale_sweep=true), warning about and closing PRs with no
non-bot activity for 30/37 days. Preserves the original policy (no-stale
and maestro exemptions, drafts included, non-bot activity timer) without
model calls or token cost.

Deletes close-stale-prs.agent.md and its compiled lock file, and updates
docs/design/pr-triage-workflows.md.

Fixes #915

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@JanKrivanek

Copy link
Copy Markdown
Member Author

Live validation (dry-run)

I dispatched the new stale-sweep job against this PR branch to observe the deterministic logic on real repo data, in dry-run mode (no writes):

Run: https://github.com/dotnet/skills/actions/runs/29899244455PR triage batch — stale sweep [dry-run], conclusion success.

What it decided (18 open PRs evaluated):

PR inactivity (non-bot) decision
#753 33 days warn (30 < inactivity ≤ 37)
#329 27 days skip (active, < 30d)
all others created ≤ 30 days ago skip (too new)

This exercises the warn threshold and the exemption/too-new filters. Per @JanKrivanek's guidance I did not post to any community PR, so this was dry-run only. No open PR currently exceeds the 37-day close threshold, so the close path had no live target today.

Sample of a real stale close

The close behavior this PR ports is the same one the (now-removed) agentic workflow performed. A recent real example is #244, auto-closed by github-actions[bot] on 2026-07-20 with exactly the closing comment this script reproduces:

(Other historical stale closes by the old bot: #263, #610.)

Notes

  • The stale_sweep / stale_max workflow_dispatch inputs are permanent — they are the manual entry point for the sweep (and are what enabled this dry-run from the branch). There is no temporary scaffolding to remove.
  • actionlint (workflow) and bash -n (script) both pass.

@JanKrivanek
JanKrivanek marked this pull request as ready for review July 22, 2026 09:17
@JanKrivanek
JanKrivanek requested a review from ViktorHofer as a code owner July 22, 2026 09:17
Copilot AI review requested due to automatic review settings July 22, 2026 09:17
@JanKrivanek
JanKrivanek enabled auto-merge (squash) July 22, 2026 09:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR replaces the agentic “close stale PRs” workflow with a deterministic stale-PR sweep integrated into the existing pr-triage-batch.yml orchestrator, reducing unreliability and eliminating model/token usage for a fully deterministic policy.

Changes:

  • Added a deterministic stale-PR sweep script and a weekly/manual stale-sweep job in pr-triage-batch.yml.
  • Removed the legacy agentic stale-PR workflows (close-stale-prs.agent.*).
  • Updated PR triage design docs to document the new stale-sweep policy and triggers.
Show a summary per file
File Description
docs/design/pr-triage-workflows.md Documents the new stale-sweep job, its policy, and how no-stale is honored.
.github/workflows/pr-triage-batch.yml Adds a weekly cron + dispatch option and a stale-sweep job to run the deterministic sweep.
.github/workflows/close-stale-prs.agent.md Deleted legacy agentic stale PR workflow definition.
.github/workflows/close-stale-prs.agent.lock.yml Deleted compiled lock workflow for the legacy agentic stale PR workflow.
.github/scripts/pr-stale-sweep.sh New deterministic stale warning/close implementation using GitHub API via gh.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 4/5 changed files
  • Comments generated: 2

Comment thread .github/scripts/pr-stale-sweep.sh Outdated
Comment thread .github/scripts/pr-stale-sweep.sh Outdated
@github-actions github-actions Bot added the waiting-on-author PR state label label Jul 22, 2026
@github-actions

Copy link
Copy Markdown
Contributor

👋 @JanKrivanek — this PR has 2 unresolved review thread(s). When you're ready, please address the feedback and push an update; the triage bot will pick up the next state automatically. (Add the no-stale label to silence further pings.)

@Evangelink Evangelink left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes because the destructive path currently fails open when activity reads fail, and the write cap can starve the stalest closures behind newer warnings. I reproduced both cases against this exact script. The existing 500-item enumeration-cap thread is also valid. I separately checked inline review replies on this repository: each reply produced a COMMENTED review with the same submitted_at, so that existing thread does not appear to be a blocker.

Comment thread .github/scripts/pr-stale-sweep.sh Outdated
Comment thread .github/scripts/pr-stale-sweep.sh Outdated
@Evangelink
Evangelink self-requested a review July 22, 2026 13:53
…, review comments

Address code review on PR #928:
- Never decide on partial data: if any activity source read fails, skip the PR
  instead of swallowing the error and risking a wrongful close.
- Apply STALE_MAX only after sorting eligible PRs by inactivity (stalest first),
  so the cap can't starve an old closure behind a newer warning.
- Enumerate open PRs with full pagination (gh api --paginate) instead of a
  fixed --limit.
- Include inline review comments as a human-activity source.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 22, 2026 17:31
@JanKrivanek

Copy link
Copy Markdown
Member Author

Review round addressed (commit 9f81276)

Reworked .github/scripts/pr-stale-sweep.sh to resolve all four review findings; each inline thread has a reply and is resolved.

  • Fail-safe reads (Evangelink, blocking): last_non_bot_activity_epoch now returns non-zero if any of the three activity reads fails; the caller skips that PR (activity read failed — skipping) instead of falling back to created_at and risking a wrongful close.
  • Sort-before-cap (Evangelink, blocking): two-phase design — collect every eligible PR, sort by inactivity descending (stalest first), then apply STALE_MAX. Closures can no longer be starved by newer warnings.
  • Full pagination (Copilot): enumeration uses gh api --paginate repos/$REPO/pulls?state=open&per_page=100 (no 500 cap).
  • Inline review comments (Copilot): pulls/{pr}/comments is now a third non-bot activity source, so a human who only leaves code-review comments isn't treated as inactive.

Verification

actionlint is green on pr-triage-batch.yml.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 4/5 changed files
  • Comments generated: 3

Comment thread docs/design/pr-triage-workflows.md
Comment thread .github/scripts/pr-stale-sweep.sh
Comment thread .github/workflows/pr-triage-batch.yml
Address Copilot re-review on PR #928:
- Validate STALE_MAX/WARN_DAYS/CLOSE_DAYS are integers and CLOSE_DAYS >
  WARN_DAYS, so a bad manual dispatch fails fast with a clear message instead
  of an opaque bash arithmetic error.
- Give the weekly stale sweep its own concurrency group so it and the hourly
  triage orchestrator don't block each other when their crons overlap (Mon 04:17).
- Correct the design doc: the orchestrator posts a one-time idempotency marker
  comment when it dispatches the malicious-code scanner (it is not comment-free).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 23, 2026 09:56
@JanKrivanek

Copy link
Copy Markdown
Member Author

Re-review round addressed (commit 3a3c9b9)

Three follow-up comments from the Copilot re-review are fixed, replied to, and resolved:

  • Numeric input validation (pr-stale-sweep.sh): STALE_MAX/WARN_DAYS/CLOSE_DAYS are now validated as integers with CLOSE_DAYS > WARN_DAYS, failing fast with a clear ::error:: instead of an opaque arithmetic error on a bad manual dispatch.
  • Concurrency isolation (pr-triage-batch.yml): the weekly stale sweep now uses its own concurrency group, so it and the hourly orchestrator don't block/queue each other when their crons overlap (Mon 04:17).
  • Doc accuracy (pr-triage-workflows.md): corrected — the orchestrator does post a one-time idempotency-marker comment when dispatching the malicious-code scanner.

Verified: bash -n clean, both bad-input cases exit 2, behavioral harness still 12/12, actionlint green, and all CI checks pass on the new head.

All review threads (7/7) are now resolved. @Evangelink — the two blocking items from your review were addressed in the prior round (9f81276); whenever you have a moment, a re-review to clear the changes-requested gate would be appreciated. Thanks!

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 4/5 changed files
  • Comments generated: 0 new

@JanKrivanek
JanKrivanek merged commit 15c3367 into main Jul 23, 2026
33 checks passed
@JanKrivanek
JanKrivanek deleted the jankrivanek-merge-close-stale-prs-into-triage branch July 23, 2026 14:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

waiting-on-author PR state label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Review 'Close Stale PRs'

3 participants