Skip to content

Require checking for duplicate issues and pending fixes before filing - #6959

Open
guirossibrum wants to merge 3 commits into
omacom:quattrofrom
guirossibrum:improve-bug-report-checklist
Open

Require checking for duplicate issues and pending fixes before filing#6959
guirossibrum wants to merge 3 commits into
omacom:quattrofrom
guirossibrum:improve-bug-report-checklist

Conversation

@guirossibrum

Copy link
Copy Markdown

Why

Issue #6648 is a concrete example of what this prevents: a report that
turned out to be a stale local plugin lock, not an Omarchy bug, closed
after a maintainer had to spend time investigating it. Neither the bug
report template nor the general-purpose contributing.md skill prompts
for a duplicate check or a version-matched reproduction before filing.

Omarchy already partially enforces this elsewhere: the diagnose-crash
agent skill's reporting.md requires searching open and closed issues,
and treats a closed-as-fixed issue that still reproduces as a regression
worth reporting, before it lets an AI agent file anything on a user's
behalf. It was missing a check for pull requests though, so this also
closes that gap there and in contributing.md, brings the same
discipline to the human-facing template, so it applies regardless of how
someone files.

What

  • bug.yml: three required checkboxes before the existing fields —
    searched issues, checked PRs, confirmed on latest release. GitHub's
    native required: true on type: checkboxes blocks submission until
    acknowledged, not just a markdown reminder someone can skip past.
  • omarchy/contributing.md: new "Verify Before Filing" section
    requiring a version-matched upstream diff, an issue search, and a PR
    search before any agent drafts a report.
  • diagnose-crash/reporting.md: added the PR-search step it was
    missing, next to its existing issue-search requirement.

Test plan

  • Validated bug.yml YAML (ruby -ryaml)
  • ./test/cli passes
  • ./test/shell has pre-existing failures on this machine from an
    unrelated missing mise Node pin; none of the failing tests reference
    any of the three files changed here (confirmed via
    grep -rl across test/), and none of these files have any existing
    test coverage

Filed by Claude Sonnet 5 via Claude Code, with @guirossibrum.

The bug report template and both AI-agent reporting skills let a report
go out without first confirming it isn't a duplicate, isn't already
fixed by an open or merged PR, or doesn't reproduce on the current
release. reporting.md already searched issues but not PRs; contributing.md
had no such step at all.

- bug.yml: three required checkboxes before the existing fields
- omarchy/contributing.md: new "Verify Before Filing" section covering
  version-matched reproduction, issue search, and PR search
- diagnose-crash/reporting.md: added the missing PR-search step next to
  its existing issue-search requirement
Copilot AI balanced review requested due to automatic review settings August 15, 2026 11:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds stronger guidance and enforcement to reduce duplicate bug reports by requiring users to check existing issues/PRs and confirm the problem exists on the latest release.

Changes:

  • Add a “Verify Before Filing” section to the Omarchy contributing guide with concrete commands for version/tag/issue/PR checks.
  • Update crash-reporting guidance to explicitly include searching pull requests.
  • Add required “Before filing” checkboxes to the GitHub bug issue template.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
default/agents/skills/omarchy/contributing.md Adds pre-filing verification steps (version, issues, PRs) to reduce duplicates.
default/agents/skills/diagnose-crash/reporting.md Extends crash reporting checklist to include PR searches.
.github/ISSUE_TEMPLATE/bug.yml Enforces pre-filing checks via required template checkboxes.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +40 to +41
in a release means "update and it's fixed," not a new bug. An open PR
means point the user at it instead of filing a duplicate.
Comment on lines +58 to +59
a release means "update and it's fixed," not a new bug; an open PR means point
the user at it instead of filing a duplicate:
Copilot review on omacom#6959 correctly flagged that a merged-but-unreleased
PR isn't actually installable via omarchy update yet, so telling users
to "update and it's fixed" is misleading. Reworded in both files to say
the fix ships in the next release, and a new report is only useful if
it still reproduces after that release.
Copilot AI review requested due to automatic review settings August 15, 2026 11:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (2)

.github/ISSUE_TEMPLATE/bug.yml:17

  • The links don’t match the checkbox claims: both queries default to showing only open items. This makes it easy for reporters to miss closed issues or merged PRs while still checking the box. Consider updating the links to include explicit guidance/queries for closed items (e.g., separate links for open vs closed, or a query that instructs users to toggle/add is:closed / is:merged).
        - label: I searched [open and closed issues](https://github.com/basecamp/omarchy/issues?q=is%3Aissue) for this and found nothing matching
          required: true
        - label: I checked [pull requests](https://github.com/basecamp/omarchy/pulls?q=is%3Apr) for an existing fix, open or merged
          required: true

default/agents/skills/omarchy/contributing.md:28

  • This example hard-codes a v<version> tag prefix, which will fail if the upstream tags don’t use a v prefix (or if omarchy version already includes it). To avoid confusion, prefer a placeholder like <tag> (derived from omarchy version) rather than forcing v<version>.
   curl -fsSL "https://raw.githubusercontent.com/basecamp/omarchy/v<version>/<path>"

Copilot review on omacom#6959 correctly noted that omarchy version's output
(e.g. 4.0.0-1) doesn't match its release tag verbatim (v4.0.0), so
constructing the tag by concatenating a hardcoded "v" prefix would
misresolve. Look up the matching tag with gh release list instead.
Copilot AI review requested due to automatic review settings August 15, 2026 11:24

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (4)

default/agents/skills/omarchy/contributing.md:25

  • The new docs emphasize validating against the installed version’s matching upstream tag, but the new issue template checkbox requires reproducing on the latest release. These are different gates and can lead users to contradictory outcomes (e.g., a bug present on their installed version but already fixed in a newer release). Consider aligning the wording: either (a) require reproduction on the latest release and adjust this section to say the same, or (b) change the template checkbox to match the ‘installed version + matching tag’ guidance and add a separate (optional) ‘also tested on latest release’ note.
1. **It reproduces against the exact installed version, on the matching
   upstream tag** — not just the default branch, since an unreleased fix
   there wouldn't apply to what's actually installed, and a fix already
   released after the installed version means it's not a live bug either.

default/agents/skills/omarchy/contributing.md:31

  • gh release list, gh issue list, and gh pr list are paginated/limited by default, so the suggested commands can miss older tags or matching results (especially for common keywords). Consider adding an explicit --limit (e.g., --limit 100) or using a search query that’s more deterministic (for releases, querying tags directly) so reporters don’t get a false ‘nothing found’ result.
   gh release list --repo basecamp/omarchy   # find the matching tag

default/agents/skills/omarchy/contributing.md:41

  • gh release list, gh issue list, and gh pr list are paginated/limited by default, so the suggested commands can miss older tags or matching results (especially for common keywords). Consider adding an explicit --limit (e.g., --limit 100) or using a search query that’s more deterministic (for releases, querying tags directly) so reporters don’t get a false ‘nothing found’ result.
   gh issue list --repo basecamp/omarchy --state all --search "<keywords>"

default/agents/skills/omarchy/contributing.md:49

  • gh release list, gh issue list, and gh pr list are paginated/limited by default, so the suggested commands can miss older tags or matching results (especially for common keywords). Consider adding an explicit --limit (e.g., --limit 100) or using a search query that’s more deterministic (for releases, querying tags directly) so reporters don’t get a false ‘nothing found’ result.
   gh pr list --repo basecamp/omarchy --state all --search "<keywords>"

Copilot AI review requested due to automatic review settings August 15, 2026 11:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (2)

default/agents/skills/omarchy/contributing.md:31

  • gh release list enumerates GitHub Releases, not necessarily all git tags. Since the text says “matching upstream tag”, this can be misleading if a tag exists without a corresponding Release. Consider either (a) changing the wording to “matching release tag” to align with the command, or (b) switching the command to one that lists tags (e.g., git ls-remote --tags or a gh api tags endpoint) so the instructions match the terminology.
   gh release list --repo basecamp/omarchy   # find the matching tag

.github/ISSUE_TEMPLATE/bug.yml:17

  • The checkbox labels say “open and closed” / “open or merged”, but the linked URLs may not actually show those states depending on GitHub’s list defaults. Consider updating the links to explicitly include state qualifiers in the query (or link to a search that clearly includes closed/merged), so the URL behavior matches the instruction text.
        - label: I searched [open and closed issues](https://github.com/basecamp/omarchy/issues?q=is%3Aissue) for this and found nothing matching
          required: true
        - label: I checked [pull requests](https://github.com/basecamp/omarchy/pulls?q=is%3Apr) for an existing fix, open or merged

@guirossibrum

Copy link
Copy Markdown
Author

Addressed both review rounds:

  • "update and it's fixed" wording (contributing.md, reporting.md): fixed — reworded to say the fix ships in the next release rather than implying it's immediately installable, since omarchy update only pulls released versions.
  • v<version> tag assumption (contributing.md): fixed — omarchy version's output (e.g. 4.0.0-1) doesn't match its release tag verbatim (v4.0.0), so the example now looks up the actual tag via gh release list instead of concatenating a hardcoded v prefix.
  • Issue/PR links defaulting to open-only (bug.yml, suppressed comment): checked against GitHub's search API directly rather than assuming either way —
    gh api "search/issues?q=repo:basecamp/omarchy+is:issue+catppuccin"   → returns closed issues (e.g. #6648, #6205)
    gh api "search/issues?q=repo:basecamp/omarchy+is:pr+catppuccin"      → returns both open and merged/closed PRs
    
    is:issue / is:pr alone, with no explicit is:open, already include closed/merged items — GitHub only defaults to open-only when there's no query at all, not when one is supplied. Left the links as-is since they already do what the checkbox text claims.

— Claude Sonnet 5 via Claude Code, with @guirossibrum.

@rdoupe

rdoupe commented Aug 30, 2026

Copy link
Copy Markdown

Worth knowing before this gets reviewed: the guidance here is right, but default/agents/skills/omarchy/contributing.md is not read by the agents it is aimed at.

So for the PR-submission path, AGENTS.md is the file that actually gets read. I opened #9095 putting the search-first guidance there. The issue-reporting path is a different story: an agent reporting a bug does load the skill, so guidance in the skill's reporting guide (post-#6718, reporting-issues.md) is well placed for that.

For this PR specifically, the bug.yml required checkbox is the strongest piece — it is enforced at filing time regardless of what anyone read, and nothing else in the set does that. Suggest keeping that and the diagnose-crash/reporting.md change, and rebasing the contributing.md part onto whatever #6718 leaves behind.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants