Skip to content

fix: sort tools deterministically to preserve prompt-cache prefix - #4

Open
sumleo wants to merge 1 commit into
codeany-ai:mainfrom
sumleo:fix/deterministic-tool-order
Open

fix: sort tools deterministically to preserve prompt-cache prefix#4
sumleo wants to merge 1 commit into
codeany-ai:mainfrom
sumleo:fix/deterministic-tool-order

Conversation

@sumleo

@sumleo sumleo commented Jun 17, 2026

Copy link
Copy Markdown

Problem

Registry.All() and Registry.Names() range over the internal map[string]types.Tool, so they return tools in Go's randomized map-iteration order. That order changes from one run to the next.

The agent loop builds the request tools[] directly from toolRegistry.All() (agent/loop.go), and the tools array sits ahead of the system block that carries cache_control. Because the bytes before the cache breakpoint change whenever the tool order shuffles, the cacheable prefix differs across otherwise-identical requests, which defeats prompt caching and causes avoidable cache misses.

Fix

Sort the returned slice by tool Name() in All(), and sort Names() with sort.Strings. This makes the order stable and deterministic. It is dependency-free (standard library sort) and does not change any public signatures.

Test

Adds tools/registry_test.go (the package had none for the registry). It registers tools in a non-sorted insertion order and asserts that All() and Names() come back sorted by name and stay identical across repeated calls.

go test ./... and go vet ./... pass locally.

@sumleo

sumleo commented Jun 18, 2026

Copy link
Copy Markdown
Author

Hi @idoubi, gentle nudge on this when you have a moment. It's a small, self-contained prompt-caching fix, and I'm happy to rebase or tweak anything if that would make review easier. Thanks for the project and your time!

@sumleo

sumleo commented Jun 27, 2026

Copy link
Copy Markdown
Author

Hi @idoubi — gentle follow-up on this one (open since mid-June). It sorts tools deterministically so the prompt-cache prefix stays byte-stable across requests. CI is green and it's a small, self-contained change. Would you have a moment to take a look, or let me know if you'd like anything adjusted? Happy to rebase. Thanks for your time and for the project!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant