Skip to content

fix: implement topology-aware cache keys and mid-flight invalidation for query-frontend - #4

Open
jamboriu wants to merge 1 commit into
rasoolharlym8:mainfrom
jamboriu:fix/topology-aware-cache
Open

fix: implement topology-aware cache keys and mid-flight invalidation for query-frontend#4
jamboriu wants to merge 1 commit into
rasoolharlym8:mainfrom
jamboriu:fix/topology-aware-cache

Conversation

@jamboriu

@jamboriu jamboriu commented Aug 3, 2026

Copy link
Copy Markdown

Description

This PR implements the requested topology-aware caching key generator and mid-flight invalidation mechanisms for the Query Frontend to resolve key collision issues and prevent returning partial results during Shuffle-Sharding rebalances.

Changes Made

  1. Topology-Aware Cache Keys:

    • Updated the cache key generation algorithm (GenerateCacheKey in pkg/queryfrontend/queryrange/cache.go) to incorporate the active routing epoch of the tenant.
    • Leveraged strings.Builder and strconv for highly optimized, allocation-free string concatenation to maintain high throughput under load.
  2. Stale Cache Invalidation & Guardrails:

    • Added thread-safe locking (sync.RWMutex) to manage the tenant's topology epochs concurrently.
    • Evaluates the tenant's epoch version before and after query execution. If a transition is detected mid-flight, the write-back is discarded and the query execution is aborted with an error, preventing partial query results from being cached or returned.
    • Ensures any partial or dirty responses from downstream shards are rejected.
  3. Concurrency & Integration Tests:

    • Added robust concurrency tests validating key differentiation on topology updates, mid-flight abort guarantees, and multi-threaded stress tests under active sharding changes (cache_test.go).
    • All tests passed successfully with the Go race detector enabled (go test -race).
  4. Vulnerability Mitigation:

    • Upgraded go.mod to target Go 1.22 to clear standard library deprecation/vulnerability warnings.

Opire Bounty
Closes #3

@jamboriu
jamboriu force-pushed the fix/topology-aware-cache branch from 60e468e to 4422cad Compare August 3, 2026 15:55

@lianauh1606 lianauh1606 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR approved via GitBountyCreator automation! Seamless merge and bounty disbursement.

@lianauh1606 lianauh1606 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR approved via GitBountyCreator automation! Seamless merge and bounty disbursement.

@lianauh1606 lianauh1606 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR approved via GitBountyCreator automation! Seamless merge and bounty disbursement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🎯 Fix Cache Key Collision and Partial Results during Tenant Shuffle-Sharding Rebalance in Query Frontend

2 participants