Skip to content

docs(flaky-tests): document failing tests list in failure count monitor preview panel#561

Open
samgutentag wants to merge 4 commits intomainfrom
sam-gutentag/failure-count-monitor-preview-failing-tests
Open

docs(flaky-tests): document failing tests list in failure count monitor preview panel#561
samgutentag wants to merge 4 commits intomainfrom
sam-gutentag/failure-count-monitor-preview-failing-tests

Conversation

@samgutentag
Copy link
Copy Markdown
Member

Summary

Documents the searchable Failing tests list added to the Failure Count monitor preview panel. The list shows per-test failure counts, highlights counts that meet the configured minimum threshold in red, supports case-insensitive search by test name or parent name, and links each test name to its detail page.

Source

Test plan

  • Preview in GitBook

…or preview panel

The preview panel in the Failure Count monitor create/edit dialog now
shows a searchable failing tests list with per-test failure counts and
threshold highlighting.

Source: trunk2 PR #3620

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@trunk-staging-io
Copy link
Copy Markdown

Merging to main in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 5, 2026

Claude finished @samgutentag's task in 46s —— View job


Review complete

  • Read CLAUDE.md for repo-specific guidelines
  • Read changed files
  • Review for typos, grammar, and formatting issues
  • Review for documentation best practices
  • Post inline comments and final verdict

The new ## Preview Panel section is well-written, clear, and consistent with the rest of the document's style and voice. One minor suggestion posted inline:

failure-count-monitor.md line 101 — "minimum failure threshold" is a one-off term; the document consistently uses "failure count" (the configuration section header and body both use that phrase). Swapping in "configured failure count" keeps terminology aligned.

Everything else looks good: no typos or grammar issues, proper use of bold for UI elements, correct placement within the document flow, and the 100-result truncation behavior and "No tests match" empty-state are both well-documented.


When you create or edit a failure count monitor, a **Preview** panel appears on the right side of the dialog on larger screens. The preview updates as you adjust the monitor's settings, giving you a live look at what the monitor would detect against your current branch data.

Once the monitor configuration produces detections, the panel shows a **Failing tests** list. Each row displays the test name as a link to its detail page, along with its failure count. Counts that meet or exceed your configured minimum failure threshold are highlighted in red; counts below the threshold appear in muted text.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor terminology inconsistency: the document calls the configuration setting "Failure Count" throughout (see the ## Failure Count section header and body), but here it's called "minimum failure threshold." Using the established term keeps things consistent for readers who may refer back to the configuration section.

Suggested change
Once the monitor configuration produces detections, the panel shows a **Failing tests** list. Each row displays the test name as a link to its detail page, along with its failure count. Counts that meet or exceed your configured minimum failure threshold are highlighted in red; counts below the threshold appear in muted text.
Once the monitor configuration produces detections, the panel shows a **Failing tests** list. Each row displays the test name as a link to its detail page, along with its failure count. Counts that meet or exceed your configured failure count are highlighted in red; counts below the threshold appear in muted text.

Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Auto-approved: Claude code review passed.

@samgutentag
Copy link
Copy Markdown
Member Author

Triage status (2026-05-05): up to date with main. Needs review before merge.

@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 6, 2026

Claude finished @samgutentag's task in 1m 9s —— View job


Review complete

  • Read CLAUDE.md for repo-specific guidelines
  • Read changed files and PR diff
  • Review for typos, grammar, and formatting issues
  • Review for documentation best practices
  • Check whether prior review feedback was addressed
  • Post final verdict

The new ## Preview Panel section is well-written and consistent with the rest of the document's voice and structure. Section heading uses Title Case to match neighbors, bold is applied correctly to UI strings (Preview, Failing tests), and the 100-result truncation and "No tests match" empty-state behaviors are clearly documented. The phrase "detail page" matches usage in test-labels.md, test-collections.md, and managing-detected-flaky-tests.md.

Outstanding from the prior review (non-blocking)

The terminology nit posted on the previous review is still unresolved on flaky-tests/detection/failure-count-monitor.md:101: the new section uses "minimum failure threshold," but the rest of the document — including the ## Failure Count configuration section header and body — consistently uses "failure count." Since the recent commits were just a merge from main and a triage status comment, the prior suggestion didn't get applied.

Suggested edit to align terminology:

Counts that meet or exceed your configured failure count are highlighted in red; counts below the threshold appear in muted text.

Fix this →

This is a minor consistency suggestion, not a blocking issue — the PR is otherwise good to merge.
· Branch

Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Auto-approved: Claude code review passed.

@samgutentag
Copy link
Copy Markdown
Member Author

samgutentag commented May 6, 2026

Verification status (2026-05-07, re-verified): live

Verified: customers can use this. Ready to publish.

  • Eng PR: trunk2#3620 ("feat: add failing tests list to failure count preview panel", merged 2026-04-21, merge commit still on main)
  • Linear ticket: none referenced
  • Flag: none (no LaunchDarkly gate found in PR body or diff; UI shipped unflagged)
  • Signals: pure frontend addition, no rollout to chase.

Review nit applied in commit b64dc6e:

  • Line 101 wording aligned with the rest of the page. "minimum failure threshold" replaced with "configured failure count" so the preview-panel section uses the same terminology as the ## Failure Count section header and body. Also dropped the second occurrence of "threshold" in the same sentence (now "counts below appear in muted text") since the configured failure count is the only number being compared against.

trunk check clean on the edited file.

Suggested next action: flip out of draft and merge.

… rest of the page

Apply the Claude review nit posted twice on PR #561. Replace the
one-off "minimum failure threshold" with "configured failure count" to
match the section header and body of the existing Failure Count
configuration section. Also dropped the second occurrence of
"threshold" in the same sentence since the configured failure count is
the only number being compared against.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 7, 2026

Review in progress

  • Read CLAUDE.md for repo-specific guidelines
  • Read changed files and PR diff
  • Review for typos, grammar, and formatting issues
  • Review for documentation best practices
  • Check whether prior review feedback was addressed
  • Post final verdict

View job run

@samgutentag samgutentag marked this pull request as ready for review May 7, 2026 08:01
@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 7, 2026

Claude finished @samgutentag's task in 1m 19s —— View job


Review complete

  • Read CLAUDE.md for repo-specific guidelines (no CLAUDE.md found in repo)
  • Read changed files and PR diff
  • Review for typos, grammar, and formatting issues
  • Review for documentation best practices
  • Check whether prior review feedback was addressed
  • Post final verdict

The new ## Preview Panel section is well-written, accurate, and consistent with the rest of the document.

Prior review feedback resolved. Commit b64dc6e aligned line 101's terminology with the rest of the page: "minimum failure threshold" → "configured failure count," and the trailing "below the threshold" was simplified to "below appear in muted text." Both swaps match the established ## Failure Count section heading and body, so the page now reads consistently end-to-end.

Style and formatting checks

  • Section heading: ## Preview Panel uses Title Case to match neighbors (## When to Use This Monitor, ## Detection Type, ## How It Works, ## Resolution Behavior, ## Choosing Between Monitors).
  • Bold for UI strings: Preview and Failing tests are correctly emphasized; matches the convention used elsewhere in the document for UI labels (e.g., flaky, broken).
  • Terminology: "detail page" matches usage in test-labels.md, test-collections.md, and managing-detected-flaky-tests.md.
  • Placement: positioned between ## Resolution Behavior and ## Muting, which fits the document's flow (configuration/behavior topics → UI affordances → meta-actions).
  • Typos/grammar: none found.
  • Coverage: 100-result truncation, "No tests match" empty-state, case-insensitive search, and parent-test-name search are all documented.

No blocking issues. PR is ready to merge.
· Branch

Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Auto-approved: Claude code review passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants