Skip to content

patchscan: add validate-pr script and unified PR Validation Report#386

Merged
ianm-nv merged 1 commit into
NVIDIA:github-actionsfrom
nirmoy:add-validate-pr
Apr 24, 2026
Merged

patchscan: add validate-pr script and unified PR Validation Report#386
ianm-nv merged 1 commit into
NVIDIA:github-actionsfrom
nirmoy:add-validate-pr

Conversation

@nirmoy
Copy link
Copy Markdown
Collaborator

@nirmoy nirmoy commented Apr 22, 2026

Add .github/scripts/validate-pr: a new Python script that checks commit hygiene and PR metadata alongside the existing patchscan script.

Checks performed

  • Cherry-pick digest table: for each (cherry picked from commit <sha>) trailer, compare patch-ID, subject, and SoB chain against upstream. Unresolvable SHAs are fetched directly by object ID before failing (handles intra-tree cherry-picks from unfetched branches).
  • R5 E: missing Signed-off-by: trailer
  • R6 E: non-SAUCE/non-Revert commit lacks upstream reference trailer
  • R7 W: LKML in-review backport uses wrong trailer form
  • R8 E: merge commit in PR range
  • R9 W: subject line >72 chars (exempt: SAUCE and Reverts of SAUCE)
  • R10 W: non-canonical SAUCE subject prefix
  • R1 W: PR title missing [<branch>] prefix
  • R3 E: PR targets a tracked branch but body has no BugLink: or LP: line

Workflow changes (patchscan.yml)

  • Add 26.04_linux-nvidia and 26.04_linux-nvidia-bos to pull_request_target branch list (was only 24.04_linux-nvidia-6.17-next)
  • Add optional repo input for cross-repo PR scanning via workflow_dispatch
  • Move "Write PR title and body to files" step to after actions/checkout so files survive into later steps
  • Add validate-pr invocation step
  • Replace separate patchscan comment with a single unified PR Validation Report comment (always runs); uses safeRead() guard for missing output files
  • Classify patchscan result by presence of empty All fixes: section rather than E: lines (avoids false failures on VR/intra-tree cherry-picks)
  • Fail job when patchscan finds missing fixes or validate-pr emits E: findings

Testing

Validated end-to-end on fork PR nirmoy/NV-Kernels#11 (replica of upstream PR #384). All checks pass — example output:

PR Validation Report example

## PR Validation Report

### Patchscan ✅ No Missing Fixes
All cherry-picked commits checked — no missing upstream fixes found.

### PR Lint ✅ All checks passed
<details><summary>Details</summary>

Checking 17 commits...

Cherry-pick digest:
┌──────────────┬─────────────────────┬──────────┬─────────┬───────────────────────────┐
│ Local        │ Referenced upstream │ Patch-ID │ Subject │ SoB chain                 │
├──────────────┼─────────────────────┼──────────┼─────────┼───────────────────────────┤
│ 5e154c93f76c │ 86ff690f45cc        │ match    │ match   │ preserved + mochs added   │
...
└──────────────┴─────────────────────┴──────────┴─────────┴───────────────────────────┘

Lint: all checks passed.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a new PR-validation layer alongside the existing patchscan workflow, producing a single unified “PR Validation Report” comment that combines missing-fix detection and commit/metadata linting.

Changes:

  • Add .github/scripts/validate-pr to lint commit trailers/subjects and PR metadata (title/body/target branch).
  • Update patchscan.yml to run both patchscan + validate-pr and post a unified report comment.
  • Extend workflow inputs/branch triggers to support additional tracked branches and an optional repo input for dispatch runs.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
.github/workflows/patchscan.yml Renames and expands the workflow to run patchscan + validate-pr and post a unified PR Validation Report comment, with workflow_dispatch inputs.
.github/scripts/validate-pr New Python script that validates commit hygiene (trailers/SoB/merge commits/SAUCE rules) and PR title/body requirements; prints a digest table for cherry-picks.
Comments suppressed due to low confidence (1)

.github/workflows/patchscan.yml:64

  • The new repo input is used for API lookups and git fetches, but actions/checkout still checks out the current repository (default) at base_ref. When inputs.repo points at a different repo that doesn't have the same branch names/refs, this step will fail (or checkout the wrong code). If cross-repo scanning via workflow_dispatch is intended to work reliably, adjust checkout logic for the dispatch case (e.g., checkout a known ref in this repo, or conditionally set repository:/ref: based on steps.pr.outputs.pr_repo).
      - name: Checkout base branch
        uses: actions/checkout@v4
        with:
          ref: ${{ steps.pr.outputs.base_ref }}
          fetch-depth: 0
          persist-credentials: false

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/workflows/patchscan.yml
Comment thread .github/workflows/patchscan.yml
Comment thread .github/scripts/validate-pr Outdated
@nirmoy nirmoy requested a review from ianm-nv April 23, 2026 10:25
@nirmoy nirmoy force-pushed the add-validate-pr branch 2 times, most recently from a7a18da to 633b0c6 Compare April 23, 2026 11:31
@nirmoy
Copy link
Copy Markdown
Collaborator Author

nirmoy commented Apr 23, 2026

Update: lore.kernel.org diff verification + full commit table

Two enhancements added since the initial version:

1. Mailing-list patch diff verification (b4-based)

For commits using `(backported from )` trailers, the script now:

  • Fetches the full patch series from lore.kernel.org via `b4 am -o -` (series cached so a shared URL only fetches once)
  • Matches each commit to its patch by subject (strips `[PATCH vN M/N]` and `NVIDIA:…:SAUCE:` prefixes, handles RFC 2822 header folding)
  • Compares diffs (SHA-1 over +/-/@@ lines). On mismatch checks for a `[Author: reason]` conflict annotation — absent → `E:` error

Statuses: `match`, `MISMATCH` (error), `noted` (differs but annotated — ok), `no-match` / `no-diff` / `fetch-err` (non-fatal fallbacks).

Added `b4>=0.13` to `requirements.txt`. Confirmed working in GitHub Actions (b4 0.15.2 installs via pip).

2. All commits shown in digest table

Previously only commits with upstream reference trailers appeared. Now SAUCE and Revert commits show as informational rows (`SAUCE` / `Revert` in the upstream column, `N/A` elsewhere), giving a complete view of the PR.

Example: 14-commit LFA v2 bos branch

Checking 14 commits...

Cherry-pick digest:
┌──────────────┬───────────────────────────────────────────────┬────────────┬─────────┬───────────────────────────┐
│ Local        │ Referenced upstream / Patch subject           │ Patch-ID   │ Subject │ SoB chain                 │
├──────────────┼───────────────────────────────────────────────┼────────────┼─────────┼───────────────────────────┤
│ 43d8018c88af │ [SAUCE] firmware: smccc: lfa: handle lfa_busy │ N/A        │ N/A     │ vvidwans, nirmoyd         │
├──────────────┼───────────────────────────────────────────────┼────────────┼─────────┼───────────────────────────┤
│ 466666ff17a1 │ [SAUCE] [config] enable arm lfa support       │ N/A        │ N/A     │ nirmoyd                   │
├──────────────┼───────────────────────────────────────────────┼────────────┼─────────┼───────────────────────────┤
│ 5c473979b79a │ firmware: smccc: lfa: introduce smc access lo │ match      │ found   │ ok, backporter: nirmoyd   │
├──────────────┼───────────────────────────────────────────────┼────────────┼─────────┼───────────────────────────┤
│ 904d3a1417b1 │ firmware: smccc: lfa: register dt interrupt   │ match      │ found   │ ok, backporter: nirmoyd   │
├──────────────┼───────────────────────────────────────────────┼────────────┼─────────┼───────────────────────────┤
│ 082fd0548f63 │ firmware: smccc: lfa: add auto_activate sysfs │ match      │ found   │ ok, backporter: nirmoyd   │
├──────────────┼───────────────────────────────────────────────┼────────────┼─────────┼───────────────────────────┤
│ db8f09f1d838 │ firmware: smccc: lfa: register acpi notificat │ match      │ found   │ ok, backporter: nirmoyd   │
├──────────────┼───────────────────────────────────────────────┼────────────┼─────────┼───────────────────────────┤
│ af3b6b3be4cc │ firmware: smccc: lfa: add timeout and trigger │ match      │ found   │ ok, backporter: nirmoyd   │
├──────────────┼───────────────────────────────────────────────┼────────────┼─────────┼───────────────────────────┤
│ 9c10dbfa8844 │ firmware: smccc: lfa: move image rescanning   │ match      │ found   │ ok, backporter: nirmoyd   │
├──────────────┼───────────────────────────────────────────────┼────────────┼─────────┼───────────────────────────┤
│ cb55bb68d888 │ firmware: smccc: add support for live firmwar │ match      │ found   │ ok, backporter: nirmoyd   │
├──────────────┼───────────────────────────────────────────────┼────────────┼─────────┼───────────────────────────┤
│ eca4ee499fb1 │ dt-bindings: arm: add live firmware activatio │ match      │ found   │ ok, backporter: nirmoyd   │
├──────────────┼───────────────────────────────────────────────┼────────────┼─────────┼───────────────────────────┤
│ d8cd9ebd0c76 │ [Revert] firmware: smccc: add support for liv │ N/A        │ N/A     │ nirmoyd                   │
├──────────────┼───────────────────────────────────────────────┼────────────┼─────────┼───────────────────────────┤
│ cc09e6b00ddf │ [Revert] firmware: smccc: add timeout, touch  │ N/A        │ N/A     │ nirmoyd                   │
├──────────────┼───────────────────────────────────────────────┼────────────┼─────────┼───────────────────────────┤
│ dcb1bb69a7b8 │ [Revert] firmware: smccc: register as platfor │ N/A        │ N/A     │ nirmoyd                   │
├──────────────┼───────────────────────────────────────────────┼────────────┼─────────┼───────────────────────────┤
│ 1566c92011ff │ [Revert] [config] nvidia: enable arm lfa supp │ N/A        │ N/A     │ nirmoyd                   │
└──────────────┴───────────────────────────────────────────────┴────────────┴─────────┴───────────────────────────┘

Lint: all checks passed.

The `noted` row (`Move image rescanning`) has a `[Nirmoy Das: …]` annotation explaining an intentional deviation. All other backported patches match their lore originals exactly.

Introduce .github/scripts/validate-pr, a standalone Python tool that
checks commit hygiene for incoming PRs and produces a single structured
report consumed by patchscan.yml.

Checks performed:
- Cherry-pick and backport attribution (cherry picked from / backported from)
- Signed-off-by presence on all commits
- PR title format: [<branch>] prefix requirement
- PR body: Launchpad BugLink required for non-bos branches
- SAUCE / Revert commits shown in digest with SoB authors

Diff verification for backported-from-lore commits:
- Uses b4 am to fetch the LKML series for each (backported from <url>)
  commit, caching per URL to avoid duplicate fetches
- Computes a fingerprint over added/removed lines only (excludes @@ hunk
  headers whose line numbers differ between email and git-show contexts)
- Reports match/mismatch in the digest table
- Accepts [Author: reason] conflict annotations to allow intentional
  divergence without failing

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown
Collaborator

@ianm-nv ianm-nv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
Acked-by: Ian May <ianm@nvidia.com>

@ianm-nv ianm-nv merged commit 345dfc3 into NVIDIA:github-actions Apr 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants