Skip to content

router: fix lora-only route rate limits#1237

Open
avinxshKD wants to merge 1 commit into
volcano-sh:mainfrom
avinxshKD:fix/router-lora-ratelimit
Open

router: fix lora-only route rate limits#1237
avinxshKD wants to merge 1 commit into
volcano-sh:mainfrom
avinxshKD:fix/router-lora-ratelimit

Conversation

@avinxshKD

Copy link
Copy Markdown
Contributor

/kind bug

What this PR does / why we need it:

Fixes rate limits for LoRA-only ModelRoutes.

Previously the router registered the limiter only with spec.modelName. For a LoRA-only route, modelName is empty, so the limiter was added under an empty key and requests using the LoRA adapter name were not limited.

This applies the same limiter to configured loraAdapters too.

Which issue(s) this PR fixes:
None

Special notes for your reviewer:
NONE

Does this PR introduce a user-facing change?:

Fix rateLimit not being applied to LoRA-only ModelRoutes.

@volcano-sh-bot

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign yaozengzeng for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the router to configure and delete rate limiters for LoRA adapters in addition to the base model, and adds a corresponding unit test. The reviewer identified a critical issue where unconditionally deleting rate limiters upon a ModelRoute deletion could leave other active routes sharing the same model or LoRA adapters unprotected, and suggested checking if the model or adapter is still registered in the store before deletion.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread pkg/kthena-router/router/router.go
@avinxshKD avinxshKD force-pushed the fix/router-lora-ratelimit branch 2 times, most recently from 3a20ba3 to a4ac907 Compare June 20, 2026 09:07
Signed-off-by: Avinash Kumar Deepak <avinash8655279@gmail.com>
@avinxshKD avinxshKD force-pushed the fix/router-lora-ratelimit branch from a4ac907 to c1a2352 Compare June 20, 2026 09:10
@avinxshKD

Copy link
Copy Markdown
Contributor Author

Hey @hzxuzhonghu @LiZhenCheng9527 added a minimal fix for LoRA-only ModelRoute rate limits.

Also covered the delete case so a limiter is not removed while another active rate-limited route still uses the same LoRA adapter. Please review when you get time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants