feature: Valkey cache integration (reuses go-redis client) by daric93 · Pull Request #1263 · gorse-io/gorse

daric93 · 2026-05-05T20:32:58Z

Summary

Adds Valkey as a cache storage backend by reusing the existing go-redis client. Valkey is wire-compatible with Redis for all commands except the TimeSeries module (TS.ADD, TS.RANGE), so instead of adding a separate client library we:

Connect to Valkey using the same go-redis client (rewriting valkey:// → redis:// for URL parsing)
Embed the existing Redis struct in a new RedisValkey type to inherit all methods unchanged
Override only AddTimeSeriesPoints and GetTimeSeriesPoints with a sorted-set + hash implementation

This keeps zero new dependencies for Valkey support.

What's included

RedisValkey type in storage/cache/redis_valkey.go — embeds Redis, overrides only time series methods
Sorted-set + hash time series: ZADD for timestamp index, HSET for values, client-side bucket aggregation
URL schemes: valkey://, valkeys://, valkey+cluster://, valkeys+cluster://
Config validation and documentation updates
Comprehensive test suite: full baseTestSuite inheritance, escape character tests, pagination tests, sorted-set TS validation against Redis
Time series benchmarks (single ingest, batch ingest, small/large/wide range queries)
CI: Valkey service container (valkey/valkey-bundle:9.1-rc2), VALKEY_URI env var, skip patterns for unit-only jobs
Docker Compose service for local development

Design decisions

Reuses go-redis — no new client dependency. Valkey speaks the Redis protocol, so go-redis works directly.
RedisValkey embeds Redis — all KV, queue, scores/search, scan, and purge operations are inherited with zero code duplication.
Sorted-set time series — Valkey lacks RedisTimeSeries. Uses ZADD (score=timestamp_ms, member=timestamp_ms) + HSET (field=timestamp_ms, value=float64) with client-side LAST-bucket aggregation matching TS.RANGE ... AGGREGATION last.
ZADD handles duplicates naturally — score update = last-write-wins, matching TimeSeries DUPLICATE_POLICY LAST.
FT.SEARCH via valkey-search — the search module commands (FT.CREATE, FT.SEARCH, FT.DROPINDEX, FT._LIST) work identically on Valkey with the valkey-search module.
Cluster mode supported via valkey+cluster:// / valkeys+cluster:// URL schemes.

Why not valkey-go or valkey-glide

Option	Issue
`valkey-glide`	Requires CGo (Rust FFI). Gorse builds with `CGO_ENABLED=0` for static `FROM scratch` binaries.
`valkey-go`	Pure Go but adds a new dependency for wire-compatible commands. Unnecessary when go-redis already works.
`go-redis` (this PR)	Zero new deps. Valkey is wire-compatible. Only time series needs a different implementation.

Testing

Inherits full baseTestSuite (TestDocument, TestPurge, TestTimeSeries, TestScanScores, etc.)
ValkeyViaRedisTestSuite — runs against real Valkey via VALKEY_URI
RedisSortedSetTSTestSuite — validates sorted-set TS implementation against Redis itself
Valkey-specific tests: escape characters, pagination with maxSearchResults override, non-existent series
Time series benchmarks: single/batch ingest, small/large/wide range queries
Additional shared tests: duplicate timestamps, empty range, multiple series isolation, bucket aggregation, batch add, empty batch

Changed files

File	Change
`storage/cache/redis_valkey.go`	New — `RedisValkey` type with sorted-set time series
`storage/cache/redis_test.go`	New test suites: `ValkeyViaRedisTestSuite`, `RedisSortedSetTSTestSuite`
`storage/cache/database_test.go`	Additional shared time series tests and benchmarks
`storage/scheme.go`	Add Valkey URL prefix constants
`storage/docker-compose.yml`	Add Valkey service (port 6380)
`.github/workflows/build_test.yml`	Add Valkey service container + env var + skip pattern
`config/config.go`	Accept Valkey prefixes in validation
`config/config.toml`	Document Valkey connection strings
`config/config_test.go`	Validate Valkey prefixes
`go.mod` / `go.sum`	Dependency updates (no new Valkey-specific deps)

Implement Valkey as a first-class cache store backend using valkey-glide Go client. This enables Gorse users to use Valkey as an alternative to Redis for the cache layer. Changes: - Add valkey://, valkeys://, valkey+cluster://, valkeys+cluster:// URL prefix constants and config validation - Implement full cache.Database interface in storage/cache/valkey.go using valkey-glide v2 client (standalone + cluster modes) - Time series implemented via Sorted Set + Hash with Go-side bucket aggregation (no TimeSeries module dependency) - Score/document operations use valkey-search FT.* commands via CustomCommand - Add integration test suite embedding baseTestSuite - Add integration report with analysis and task breakdown Signed-off-by: Daria Korenieva <daric2612@gmail.com>

Add valkey/valkey-bundle:unstable (valkey-search 1.2.0+ with full-text search support) to storage/docker-compose.yml on port 6380. Update default test URI to valkey://127.0.0.1:6380/. Signed-off-by: Daria Korenieva <daric2612@gmail.com>

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

…/gorse-io-gorse into feature/valkey-cache-integration

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

…er ops, handle username in URL Signed-off-by: Daria Korenieva <daric2612@gmail.com>

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

…ix TAG escaping: new escapeTag() covers all special chars (| * { } etc.) to prevent query injection via user-controlled values - Fix CROSSSLOT error: delete keys one-at-a-time in cluster mode via vDel() - Fix race condition: remove non-atomic Exists+HSet in UpdateScores - Add max iteration guard (100) to DeleteScores loop - Extract wrapper methods (vDel, vHSet, vZAdd, vHGetAll) to reduce cluster/standalone branching duplication - Extract groupTimeSeriesPoints() helper to DRY AddTimeSeriesPoints Signed-off-by: Daria Korenieva <daric2612@gmail.com>

…/gorse-io-gorse into feature/valkey-cache-integration # Conflicts: # storage/cache/valkey.go

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

…s - Add valkey/valkey-bundle:unstable service on port 6380 to unit_test job - Add VALKEY_URI env var for Linux unit tests - Add TestValkey to -skip pattern for macOS and Windows jobs (no Valkey service available on those runners) Signed-off-by: Daria Korenieva <daric2612@gmail.com>

…glide requires CGO (Rust FFI core) and cannot build with CGO_ENABLED=0, which gorse uses for static Docker binaries. This is a confirmed upstream limitation (valkey-io/valkey-glide#4253). valkey-go is the official pure-Go Valkey client from the same org. It supports all needed commands (FT.SEARCH, HSET, ZADD, SCAN, etc.), works with CGO_ENABLED=0, and simplifies the code by using a single Client interface for both standalone and cluster modes. Changes: - Replace valkey-glide/go/v2 with valkey-io/valkey-go in go.mod - Rewrite storage/cache/valkey.go using valkey-go command builder API - Eliminate all if-isCluster branching (valkey-go handles routing) - Use DoMulti for pipeline batching instead of standalone batch - Update tests to use valkey-go client for FLUSHDB - Update URL parsers to return plain strings instead of glide types Signed-off-by: Daria Korenieva <daric2612@gmail.com>

…/gorse-io-gorse into feature/valkey-cache-integration

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

… offset in SearchScores, clarify UpdateScores pagination - Set(): use DoMulti to pipeline all SET commands in a single round-trip - AddScores(): use DoMulti to pipeline all HSET commands - SearchScores(): pass begin as LIMIT offset and (end-begin) as count instead of fetching from 0 and slicing in Go - UpdateScores(): add clarifying comments for the first-page re-fetch optimization that avoids pagination drift Signed-off-by: Daria Korenieva <daric2612@gmail.com>

zhenghaoz

Too many duplicate codes comparing to Redis backend. Please fix all tests. Could you reuse Redis client, but implement special timeseries interface if Valkey is detected?

…nt SetupSuite panic - Fix wrong variable mapping in parseValkeyURL test: was assigning username to password variable (skipped port+password instead of port+username) - Use suite.Require() in ValkeyTestSuite.SetupSuite to fail fast instead of panicking on nil pointer when service is unavailable Signed-off-by: Daria Korenieva <daric2612@gmail.com>

- Add RedisValkey type embedding Redis, overriding AddTimeSeriesPoints and GetTimeSeriesPoints with sorted-set + hash + Go-side aggregation - Register valkey:// prefixes via go-redis (wire-compatible) - Disable old standalone valkey.go/valkey_test.go with //go:build ignore - Add comprehensive time series tests to baseTestSuite - Add ValkeyViaRedisTestSuite and RedisSortedSetTSTestSuite - Add EXPERIMENT_REPORT.md with context and next steps

- Embed Redis struct in RedisValkey, inherit all methods unchanged - Override AddTimeSeriesPoints/GetTimeSeriesPoints with sorted-set + hash - Register valkey://, valkeys://, valkey+cluster://, valkeys+cluster:// schemes - Remove standalone valkey.go (~839 lines) and valkey-go dependency - Add TestValkeyViaRedis and TestRedisSortedSetTS test suites

…ve error handling - Fix CI service port: expose container port 6379 (not 6380) for Valkey - Pin valkey-bundle image to 9.1-rc2 (includes search 1.2.0) - Add Init() override with clear error for missing valkey-search module - Return empty slice instead of nil from GetTimeSeriesPoints - Add TODO for OTel semconv.DBSystemValkey - Document ZRangeByScore pagination limitation

daric93 · 2026-05-08T18:54:08Z

Done! I've reworked the implementation based on your feedback:

Reuses the go-redis client — no more valkey-go dependency. Since Valkey is wire-compatible with Redis, go-redis connects directly (we just rewrite valkey:// → redis:// for URL parsing).
RedisValkey embeds Redis — inherits all KV, queue, scores/search, scan, and purge methods with zero code duplication.
Only overrides time series — AddTimeSeriesPoints and GetTimeSeriesPoints use a sorted-set + hash implementation since Valkey lacks the RedisTimeSeries module.

The duplicate code is eliminated. All tests pass against both Redis (with TimeSeries) and Valkey (with valkey-search).

daric93 · 2026-05-20T20:51:33Z

@zhenghaoz Please take a look when you have time - I have reworked the implementation based on your feedback.

…redis:// URL scheme for Valkey connections. Detect Valkey at connect time via INFO server (server_name:valkey) and return RedisValkey with sorted-set time series when detected. - Remove valkey://, valkeys://, valkey+cluster://, valkeys+cluster:// prefixes - Add isValkey() detection using INFO server response - RedisValkey embeds Redis, overrides only AddTimeSeriesPoints/GetTimeSeriesPoints - All other operations inherited unchanged (wire-compatible) Signed-off-by: Daria Korenieva <daric2612@gmail.com>

daric93 · 2026-05-21T17:16:52Z

@zhenghaoz Pushed the rework — removed all valkey:// prefixes, now auto-detects via INFO server. Ready for another look.

…with RedisClientName)

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

daric93 · 2026-05-29T17:05:10Z

@zhenghaoz Ready for review whenever you get a chance.

daric93 · 2026-06-08T16:18:13Z

@zhenghaoz PR is ready for review whenever you get a chance.

daric93 · 2026-06-19T18:45:32Z

@zhenghaoz could you review this when you have a moment?

daric93 added 17 commits April 29, 2026 08:22

fixes

10eda26

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

Merge branch 'feature/valkey-cache-integration' of github.com:daric93…

7a5da43

…/gorse-io-gorse into feature/valkey-cache-integration

benchmark tests

973737d

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

updated report

9f1dd97

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

fix: valkey cache - pipeline Set, fix ScanScores pattern, batch clust…

bc7c89b

…er ops, handle username in URL Signed-off-by: Daria Korenieva <daric2612@gmail.com>

fixes

282b2fa

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

Merge branch 'feature/valkey-cache-integration' of github.com:daric93…

35cba84

…/gorse-io-gorse into feature/valkey-cache-integration # Conflicts: # storage/cache/valkey.go

fixes

9f2fa68

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

clean up

16108dd

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

Merge branch 'feature/valkey-cache-integration' of github.com:daric93…

84cb864

…/gorse-io-gorse into feature/valkey-cache-integration

clean up

c823a65

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

zhenghaoz requested changes May 7, 2026

View reviewed changes

daric93 added 5 commits May 7, 2026 11:06

clean up

eb3121f

daric93 changed the title ~~feature: Valkey cache integration~~ feature: Valkey cache integration (reuses go-redis client) May 8, 2026

merge upstream/master into feature/valkey-cache-integration

30cd0b0

zhenghaoz requested changes May 21, 2026

View reviewed changes

Comment thread storage/scheme.go Outdated

daric93 added 2 commits May 27, 2026 10:08

Merge master into feature/valkey-cache-integration (resolve conflict …

ba20d68

…with RedisClientName)

chore: update valkey image from 9.1-rc2 to latest

b30bffa

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feature: Valkey cache integration (reuses go-redis client)#1263

feature: Valkey cache integration (reuses go-redis client)#1263
daric93 wants to merge 26 commits into
gorse-io:masterfrom
daric93:feature/valkey-cache-integration

daric93 commented May 5, 2026 •

edited

Loading

Uh oh!

zhenghaoz left a comment

Uh oh!

daric93 commented May 8, 2026

Uh oh!

daric93 commented May 20, 2026

Uh oh!

Uh oh!

daric93 commented May 21, 2026

Uh oh!

daric93 commented May 29, 2026

Uh oh!

daric93 commented Jun 8, 2026

Uh oh!

daric93 commented Jun 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

daric93 commented May 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What's included

Design decisions

Why not valkey-go or valkey-glide

Testing

Changed files

Uh oh!

zhenghaoz left a comment

Choose a reason for hiding this comment

Uh oh!

daric93 commented May 8, 2026

Uh oh!

daric93 commented May 20, 2026

Uh oh!

Uh oh!

daric93 commented May 21, 2026

Uh oh!

daric93 commented May 29, 2026

Uh oh!

daric93 commented Jun 8, 2026

Uh oh!

daric93 commented Jun 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

daric93 commented May 5, 2026 •

edited

Loading