Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions ai-development/claude-code-plugins.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -58,11 +58,14 @@ Workflow logic. Anything that's "how Claude Code should approach a task" lives h
<Card title="ai-assistant-instructions" icon="book" href="/ai-development/ai-assistant-instructions">
The rules layer these plugins operate within.
</Card>
<Card title="claude-code-routines" icon="clock" href="https://github.com/JacobPEvans/claude-code-routines">
<Card title="/ship and /finalize-pr" icon="ship" href="/ai-development/skills/ship-and-finalize">
The full skill lifecycle — phases, exit modes, known reliability gaps.
</Card>
<Card title="claude-code-routines" icon="clock" href="/automation/scheduled-routines/claude-code-routines">
Scheduled Claude.ai routines — the cron side of the same ecosystem.
</Card>
<Card title="ai-workflows" icon="github" href="https://github.com/JacobPEvans/ai-workflows">
The Copilot equivalent — reusable agentic workflows.
<Card title="ai-workflows" icon="github" href="/automation/cloud-pipelines/ai-workflows">
Reusable GitHub Actions workflows that drive the cloud half of the pipeline.
</Card>
<Card title="Source on GitHub" icon="github" href="https://github.com/JacobPEvans/claude-code-plugins">
Plugins, hooks, full README.
Expand Down
75 changes: 75 additions & 0 deletions ai-development/skills/codeql-resolution.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
---
title: "CodeQL resolution"
description: "CodeQL alerts are a separate gate from CI checks. /resolve-codeql is the skill that closes them automatically so /ship can complete."
tier: 2
---

> CodeQL is not in `statusCheckRollup`. If you only check CI, you miss the CodeQL gate. `/resolve-codeql` fixes that.

CodeQL is GitHub's static-analysis security scanner. It runs on every PR via the `Analyze (actions)` or `Analyze (javascript)` jobs (depending on what's enabled for the repo). When it finds a violation, it posts a **code-scanning alert** that blocks merge — but **does not** show up in `gh pr checks` or in the `statusCheckRollup` of a `gh pr view --json` query. Branch protection treats the alert as a separate signal.

## Why this matters for `/ship`

If `/finalize-pr` only queried `statusCheckRollup`, a PR could look ready (`statusCheckRollup.state: SUCCESS`) and still be blocked at merge time by an open CodeQL alert. The skill's Phase 3 explicitly runs a second, separate gate for CodeQL — using the REST code-scanning API, not GraphQL.

```bash
gh api repos/<owner>/<repo>/code-scanning/alerts \
--jq '[.[] | select(.state == "open")] | length'
```

If the result is non-zero, the PR is **not** ready, regardless of what CI says.

## What `/resolve-codeql` does

`/resolve-codeql fix` is the auto-resolution skill invoked by `/finalize-pr` Phase 2 when alerts exist:

<Steps>
<Step title="Enumerate alerts">
Lists all open alerts for the repo, with rule ID, severity, and file:line.
</Step>
<Step title="Classify">
For each alert: is it a real vulnerability, an intentional pattern that should be suppressed, or a false positive?
</Step>
<Step title="Fix or suppress">
Real → patch the code with a fix that addresses the rule. Intentional → add a CodeQL inline suppression with a justification comment. False positive → dismiss via the API with `"reason": "false positive"`.
</Step>
<Step title="Push the fix commit">
Signed via the App installation token, attributed to `JacobPEvans-claude[bot]`.
</Step>
<Step title="Re-run CodeQL">
The push triggers a fresh CodeQL run automatically. Wait for it to complete before re-querying alert state.
</Step>
</Steps>

## Why GraphQL doesn't show CodeQL state

The GraphQL `pullRequest` schema's `statusCheckRollup` aggregates check runs and status contexts. CodeQL alerts live in a separate `repository.codeScanningAlerts` connection — they're attached to the repo's default branch baseline, not to the PR's check run set.

A canonical query for the PR-readiness gate (from [`gh-cli-patterns`](https://github.com/JacobPEvans/claude-code-plugins/blob/main/github-workflows/skills/gh-cli-patterns/SKILL.md)) reads `statusCheckRollup` for CI, then does a **separate** REST call for CodeQL. Both must be clean for a PR to be considered ready.

## Async timing

CodeQL runs are slower than most CI checks — often 2–5 minutes after a push. This timing mismatch is why `/finalize-pr` Phase 2.6 (the fix in flight) introduces an explicit wait-for-async-checks step: poll at 30-second intervals up to 5 minutes, only advance to Phase 3 when all known check kinds have a terminal status. Without this, `/finalize-pr` could treat a still-running CodeQL run as a hard failure and bail out.

## What `/resolve-codeql` will NOT do

- **Won't dismiss real alerts as false positive.** Dismissals require justification and are reserved for genuine FPs.
- **Won't disable a CodeQL query.** Adjusting the CodeQL config is a deliberate human change.
- **Won't loop indefinitely.** Capped at 3 fix attempts per PR per invocation — beyond that, surfaces the unfixed alerts for a human.

## Where to go next

<CardGroup cols={2}>
<Card title="/ship and /finalize-pr" icon="ship" href="/ai-development/skills/ship-and-finalize">
The orchestrator that invokes `/resolve-codeql` as part of Phase 2.
</Card>
<Card title="claude-code-plugins" icon="plug" href="/ai-development/claude-code-plugins">
The plugin home for `/resolve-codeql` and the rest of the toolchain.
</Card>
<Card title="Source: codeql-resolver" icon="github" href="https://github.com/JacobPEvans/claude-code-plugins/tree/main/codeql-resolver">
The full plugin source.
</Card>
<Card title="GitHub CodeQL docs" icon="book" href="https://docs.github.com/en/code-security/code-scanning">
The upstream CodeQL documentation for query authoring and config.
</Card>
</CardGroup>
37 changes: 37 additions & 0 deletions ai-development/skills/overview.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
---
title: "Skills"
description: "Local Claude Code slash-commands that pick up where the cloud pipeline stops. Use them from a session when you are the one iterating on a PR."
tier: 1
---

> Cloud pipelines get a PR opened, reviewed, and CI'd without you. Skills are what you run from a Claude Code session when you are the one driving a PR to merge.

A **skill** in this catalog is a slash-command shipped by [`JacobPEvans/claude-code-plugins`](/ai-development/claude-code-plugins). The plugin repo carries the source — each skill is a folder with `SKILL.md`, prompts, and supporting scripts; this section carries the human-readable narrative for the load-bearing ones.

Skills are the local-execution counterpart to the [cloud pipelines](/automation/cloud-pipelines/overview) and [scheduled routines](/automation/scheduled-routines/overview). When the cloud finishes the job, you do not run a skill. When the cloud opens a draft PR but cannot drive it all the way (CodeQL alert needs human triage, review threads need negotiation, CI flake needs a retry strategy), you `/ship` it from a session.

## What is documented here

| Skill | Use when |
| --- | --- |
| [`/ship` and `/finalize-pr`](/ai-development/skills/ship-and-finalize) | You have a PR open and want it driven to a fully-mergeable state in one command |
| [`/resolve-codeql`](/ai-development/skills/codeql-resolution) | A PR is blocked on a CodeQL alert that does not show up in `gh pr checks` |

More skills exist in `claude-code-plugins`; only the ones that participate in the issue → mergeable PR flow are written up here. The full catalog (commands, hooks, agents, skills) lives on the [claude-code-plugins page](/ai-development/claude-code-plugins).

## Where to go next

<CardGroup cols={2}>
<Card title="claude-code-plugins" icon="plug" href="/ai-development/claude-code-plugins">
The source repo and full plugin catalog.
</Card>
<Card title="Cloud pipelines" icon="github" href="/automation/cloud-pipelines/overview">
The event-triggered half — what skills do not have to do because the cloud already handled it.
</Card>
<Card title="Issue to PR pipeline" icon="diagram-project" href="/automation/issue-to-pr-pipeline">
Where each skill plugs into the broader flow.
</Card>
<Card title="Commit conventions" icon="code-commit" href="/conventions/commit-conventions">
The voice and commit-shape rules every skill applies before pushing.
</Card>
</CardGroup>
126 changes: 126 additions & 0 deletions ai-development/skills/ship-and-finalize.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
---
title: "/ship and /finalize-pr"
description: "The local Claude Code skill that drives a PR to a fully-mergeable state. Phases, exit modes, and the reliability gaps being closed."
tier: 2
---

> One command. Commit, push, open the PR, fix CodeQL, resolve threads, fix CI, verify everything is green. Never merges. That's a human decision.

`/ship` is the local escape hatch for the cloud pipeline. The cloud pipeline ([ai-workflows](/automation/cloud-pipelines/ai-workflows), [claude-code-routines](/automation/scheduled-routines/claude-code-routines)) gets a PR opened and reviewed without you. `/ship` is what you run from a Claude Code session when **you** are the one iterating on a PR and want it driven to a ready-to-merge state in one command.

## What `/ship` does

`/ship` is a thin orchestrator. It detects uncommitted changes (commits them), discovers recently-created PRs, then for each PR invokes `/finalize-pr` — which is where the real work happens.

```text
/ship
├─ Step 0: verify git repo
├─ Step 1: commit any uncommitted changes; discover PRs in scope
├─ Step 1.5: build a context brief from the conversation
├─ Step 2: invoke /finalize-pr per PR, sequentially
└─ Step 3: re-verify each PR with live GitHub state before reporting
```

## What `/finalize-pr` does

`/finalize-pr` is the loop. Five phases per PR:

{/* Shape: linear chain. Boundary crossings: 0. Ranks: 5×1. */}
{/* Aspect: ~3:1 (LR). Pass. */}

```mermaid
%%{init: {'theme':'base','look':'handDrawn','themeVariables':{'fontFamily':'Geist','fontSize':'14px','primaryColor':'#102937','primaryTextColor':'#F4EFE6','primaryBorderColor':'#4FB3A9','lineColor':'#4FB3A9','secondaryColor':'#0B1D2A','tertiaryColor':'#1A2A38','clusterBkg':'rgba(79,179,169,0.08)','clusterBorder':'#4FB3A9'}}}%%
flowchart LR
P1([1. Discover])
P2([2. Resolve loop])
P3{3. Verify gate}
P4([4. Update metadata])
P5([5. Report ready])

P1 --> P2 --> P3 --> P4 --> P5
P3 -.->|gate fails| P2

classDef ai fill:#102937,stroke:#E06B4A,stroke-width:2px,color:#F4EFE6;
classDef auto fill:#102937,stroke:#F4EFE6,stroke-width:1.5px,color:#F4EFE6;
classDef gate fill:#102937,stroke:#E06B4A,stroke-width:2.5px,color:#F4EFE6;

class P1,P2,P4 ai
class P5 auto
class P3 gate

linkStyle 0,1,2,3 stroke:#4FB3A9,stroke-width:2px;
linkStyle 4 stroke:#E06B4A,stroke-width:1.5px,stroke-dasharray:4 3;
```

| Phase | What it owns |
| --- | --- |
| 1. Discover | Resolve which PR(s) to act on (current branch, all, or org-wide) |
| 2. Resolve loop | Start CI monitor in background; in parallel: invoke `/resolve-codeql fix`, `/resolve-pr-threads`, merge-conflict resolution; once CI completes, run `/simplify` once on the cumulative changes |
| 3. Verify gate | Re-query live GitHub state — `state`, `mergeable`, `mergeStateStatus`, `reviewDecision`, `statusCheckRollup.state`, all `reviewThreads.isResolved`, CodeQL alert count |
| 4. Update metadata | A subagent updates PR title, description, linked issues |
| 5. Report ready | Emit the canonical PR status block; wait for human merge |

## What it never does

- **Never merges.** Merge is always a human decision.
- **Never approves.** No auto-approval of PRs.
- **Never crosses org boundaries** in org-wide mode.
- **Never bypasses branch protection.**

## Subagents in the call chain

```text
/ship → /finalize-pr → /resolve-codeql fix
→ /resolve-pr-threads → superpowers:receiving-code-review
→ /simplify
→ haiku subagent (PR metadata update)
→ background CI monitor (Task tool)
```

Each subagent runs scoped to its task and reports back. The orchestrator re-verifies live state — never trusts a subagent's self-report as ground truth.

## Known reliability gaps

`/ship` has been ending prematurely without leaving PRs fully mergeable. Root causes traced to the current `SKILL.md` files:

| Failure mode | Why it happens |
| --- | --- |
| Exits "blocked" when a fix could have continued | Phase 3 → Phase 2 loop is prose with no enforced iteration counter — subagents treat it as advisory |
| Treats async-pending checks as failures | Phase 3 aborts on `statusCheckRollup.state ≠ SUCCESS`, but Phase 2 can't fix a pending check |
| Silent subagent failure passes through | After `/resolve-codeql` and `/resolve-pr-threads`, skill advances without re-querying state to confirm fixes landed |
| CI monitor dies silently | Background Task agent can fail without propagating |
| All Phase 3 aborts treated identically | Some failures are agent-fixable, some are wait, some require humans (`REVIEW_REQUIRED`) |

Fix in flight: explicit iteration counter (cap 5), wait-for-async-checks phase, post-fix verification of subagent work, CI-monitor failure fallback, and a failure-mode taxonomy that maps each `mergeStateStatus`/`reviewDecision` to a handler. After the fix, every `/ship` invocation reports one of three categories with a specific reason — never silent half-finalization:

- **Ready to merge** — all gates clean
- **Ready except human gate** — only `REVIEW_REQUIRED` remaining
- **Ship aborted** — specific gate stuck with manual-action suggestion

Tracked as a PR against [`claude-code-plugins`](https://github.com/JacobPEvans/claude-code-plugins).

## When to use `/ship` vs the cloud pipeline

| Situation | Use |
| --- | --- |
| Issue is filed, you want a PR drafted without thinking | [`ai-workflows`](/automation/cloud-pipelines/ai-workflows) (the cloud pipeline triggered on issue open) |
| You're editing a PR locally and want CI/threads/CodeQL handled in one command | `/ship` from your Claude Code session |
| You want a daily org-wide sweep | [`claude-code-routines`](/automation/scheduled-routines/claude-code-routines) (cron-scheduled) |
| You want to know if a PR is *really* ready to merge | `/finalize-pr <N>` directly — gives you the canonical gate output without re-running fixes |

## Where to go next

<CardGroup cols={2}>
<Card title="CodeQL resolution" icon="shield-check" href="/ai-development/skills/codeql-resolution">
Why CodeQL is fixed separately from CI checks, and what `/resolve-codeql` does.
</Card>
<Card title="claude-code-plugins" icon="plug" href="/ai-development/claude-code-plugins">
The plugin home for `/ship`, `/finalize-pr`, and all the supporting skills.
</Card>
<Card title="Source: ship/SKILL.md" icon="github" href="https://github.com/JacobPEvans/claude-code-plugins/blob/main/github-workflows/skills/ship/SKILL.md">
The full skill definition.
</Card>
<Card title="Source: finalize-pr/SKILL.md" icon="github" href="https://github.com/JacobPEvans/claude-code-plugins/blob/main/github-workflows/skills/finalize-pr/SKILL.md">
Phases, gates, and the merge prohibition.
</Card>
</CardGroup>
2 changes: 1 addition & 1 deletion architecture/ai-pipeline.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ Every other PAL tool has a native Claude Code equivalent; for single-model calls
</Card>
</CardGroup>

See [AI Development · Overview](/ai-development/overview) for what each one does in detail.
See [AI Development · Overview](/ai-development/overview) for what each one does in detail. For the actual triggers, callers, and skill mechanics that drive this pipeline end-to-end, see [Automation · Overview](/automation/overview).

## Observability layer

Expand Down
Loading
Loading