Skip to content

resource_group client: narrow degraded fallback conditions for GetResourceGroup#11009

Open
ystaticy wants to merge 4 commits into
tikv:masterfrom
ystaticy:rg_degraded_RUSettings
Open

resource_group client: narrow degraded fallback conditions for GetResourceGroup#11009
ystaticy wants to merge 4 commits into
tikv:masterfrom
ystaticy:rg_degraded_RUSettings

Conversation

@ystaticy

@ystaticy ystaticy commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: ref #11010

What is changed and how does it work?

Refine the degraded fallback behavior of GetResourceGroup when degradedRUSettings is configured.

Currently, degraded fallback is too broad and may hide original errors from RM. We should only use degraded fallback for transient RM failures, while preserving original errors for non-retryable cases.

Expected behavior:

  • transient RM unavailability -> degraded group
  • RM recovery -> real group
  • resource group not found -> original error
  • caller cancellation/deadline -> original context error
  • generic non-retryable error -> original error
  • nonexistent SwitchGroup target -> do not switch

It is also helpful to preserve the underlying error from ErrClientGetResourceGroup so upper layers can correctly inspect errors such as context.Canceled and context.DeadlineExceeded.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Code changes

Side effects

  • Possible performance regression
  • Increased code complexity
  • Breaking backward compatibility

Related changes

Release note

None.

Summary by CodeRabbit

Summary by CodeRabbit

  • Bug Fixes

    • Improved resource group error reporting by exposing the most relevant underlying failure cause.
    • Preserved caller Canceled and DeadlineExceeded errors reliably, including correct handling when the server returns gRPC deadline failures.
    • Refined degraded-mode fallback to trigger only for retryable, appropriate failures, with better handling for “not found” and permission-denied cases.
  • Tests

    • Expanded automated coverage for degraded fallback behavior, error classification, and context deadline/cancel preservation.

Signed-off-by: ystaticy <y_static_y@sina.com>
@ti-chi-bot ti-chi-bot Bot added do-not-merge/needs-linked-issue dco-signoff: yes Indicates the PR's author has signed the dco. release-note-none Denotes a PR that doesn't merit a release note. labels Jul 15, 2026
@ti-chi-bot

ti-chi-bot Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign cabinfeverb for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Jul 15, 2026
@coderabbitai

coderabbitai Bot commented Jul 15, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 7c5bd9ec-164a-460a-a80a-b8b3918fe406

📥 Commits

Reviewing files that changed from the base of the PR and between d847878 and 46586c5.

📒 Files selected for processing (2)
  • client/resource_manager_client.go
  • client/resource_manager_client_test.go

📝 Walkthrough

Walkthrough

The change preserves underlying resource-group errors, maps gRPC cancellation and deadline statuses to context errors, and updates controller fallback decisions to classify errors before creating degraded resource groups.

Changes

Resource group error handling

Layer / File(s) Summary
Error contract and cause preservation
client/errs/errno.go
ErrClientGetResourceGroup stores an underlying error, derives its display cause with fallbacks, and supports unwrapping.
Client error normalization and validation
client/resource_manager_client.go, client/resource_manager_client_test.go
GetResourceGroup preserves caller cancellation and deadline errors while retaining injected gRPC causes; tests verify both behaviors.
Controller fallback selection and coverage
client/resource_group/controller/global_controller.go, client/resource_group/controller/global_controller_test.go
Controller helpers unwrap and classify errors before selecting degraded fallback; tests cover transient, missing-group, cancellation, deadline, permission, switching, recovery, and unconfigured-degraded cases.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant GlobalController
  participant ResourceManagerClient
  participant DegradedResourceGroup
  Caller->>GlobalController: request resource-group controller
  GlobalController->>ResourceManagerClient: GetResourceGroup(context)
  ResourceManagerClient-->>GlobalController: wrapped error
  GlobalController->>GlobalController: normalize and classify error
  GlobalController->>DegradedResourceGroup: create degraded group when eligible
  GlobalController-->>Caller: degraded controller or normalized error
Loading

Possibly related issues

  • tikv/pd issue 11010 — Covers narrowed degraded fallback logic and preservation of underlying GetResourceGroup errors.

Suggested reviewers: bufferflies

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise and accurately summarizes the main change to GetResourceGroup degraded fallback behavior.
Description check ✅ Passed The description follows the template, includes the issue reference, change summary, checklist, and release note section.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@client/resource_group/controller/global_controller_test.go`:
- Around line 342-345: Update the newController helper to accept the subtest
testing.T and create a local require.New(t) within each invocation instead of
using the parent re. Update every t.Run call that invokes newController to pass
its subtest t, preserving the existing controller creation and assertion
behavior.

In `@client/resource_group/controller/global_controller.go`:
- Around line 579-580: Update the error handling around
normalizeGetResourceGroupError to check context.Canceled and
context.DeadlineExceeded on the original error before normalization. Then
classify the unwrapped gRPC status so codes.DeadlineExceeded reaches the
existing degraded-mode path, while preserving the current cancellation handling.

In `@client/resource_manager_client.go`:
- Around line 162-173: Update the error mapping in
client/resource_manager_client.go:162-173 around the resource-group client
method to convert codes.DeadlineExceeded to ctx.Err() only when the caller
context is done; otherwise preserve the original gRPC deadline error. Ensure
client/resource_group/controller/global_controller.go:579-588 continues routing
server-generated deadline errors through the degraded fallback, and add coverage
in client/resource_group/controller/global_controller_test.go:397-425 for a
server-returned codes.DeadlineExceeded case.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 80d51619-73e1-4ed0-9026-61aacf4a2b6e

📥 Commits

Reviewing files that changed from the base of the PR and between 6a2787b and d797561.

📒 Files selected for processing (5)
  • client/errs/errno.go
  • client/resource_group/controller/global_controller.go
  • client/resource_group/controller/global_controller_test.go
  • client/resource_manager_client.go
  • client/resource_manager_client_test.go

Comment thread client/resource_group/controller/global_controller_test.go Outdated
Comment thread client/resource_group/controller/global_controller.go Outdated
Comment thread client/resource_manager_client.go
@codecov

codecov Bot commented Jul 15, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 78.33333% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.21%. Comparing base (5c811b2) to head (d797561).
⚠️ Report is 22 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff            @@
##           master   #11009    +/-   ##
========================================
  Coverage   79.20%   79.21%            
========================================
  Files         541      541            
  Lines       75450    76039   +589     
========================================
+ Hits        59762    60236   +474     
- Misses      11456    11549    +93     
- Partials     4232     4254    +22     
Flag Coverage Δ
unittests 79.21% <78.33%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ystaticy ystaticy changed the title change getResourceGroup ,degraded fallback conditions resource_group client: narrow degraded fallback conditions for GetResourceGroup Jul 15, 2026
ystaticy added 3 commits July 15, 2026 17:26
Signed-off-by: ystaticy <y_static_y@sina.com>
Signed-off-by: ystaticy <y_static_y@sina.com>
Signed-off-by: ystaticy <y_static_y@sina.com>
@ystaticy

Copy link
Copy Markdown
Contributor Author

/test pull-unit-test-next-gen-2

@ystaticy

Copy link
Copy Markdown
Contributor Author

/retest-required

1 similar comment
@ystaticy

Copy link
Copy Markdown
Contributor Author

/retest-required

@ystaticy

Copy link
Copy Markdown
Contributor Author

/retest

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dco-signoff: yes Indicates the PR's author has signed the dco. release-note-none Denotes a PR that doesn't merit a release note. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant