feat(binding-mcp): mcp · proxy cache option (#1737)

[jfallows]

Closes #1737.

Summary

Adds options.cache to the mcp · proxy binding. When configured, the proxy attaches a single per-binding hydrate session to its downstream exit, populates a shared store with tools/list, resources/list, and prompts/list payloads, and serves those list calls from cache without round-tripping downstream. Folds what was originally proposed as a separate mcp · cache kind into an option on the existing proxy — every original topology has an exact equivalent in the folded model, and the engine kind: cache prerequisite drops out.

Config

options:
  cache:
    store: <store-name>      # required
    ttl:                     # optional; per kind
      tools: PT1S
      resources: PT2S
      prompts: PT3S
    authorization:           # optional; max one entry (schema-enforced)
      <guard-name>:
        credentials: "<value>"

Runtime behaviour

• Hydrate. At binding attach, the proxy opens one lifecycle stream to downstream (session id from MCP_SESSION_ID) and issues per-kind list BEGINs in parallel. Each kind's response body is written to the configured store under keys tools / resources / prompts.
• Lifecycle gating. Agent lifecycle BEGIN replies are deferred via signaler.signalNow (delivered to the agent's reply-direction stream — the engine registers binding consumers in throttles[replyId], where SignalFW is dispatched) until every configured kind has settled. By the time an agent sees lifecycle initialized, every list call hits a populated cache.
• Authorization. When options.cache.authorization is set, attach resolves the single guard, calls guard.reauthorize(traceId, binding.id, 0L, credentials) once, and stamps the resulting token onto every outbound BEGIN/END/WINDOW on the hydrate session and its list streams.
• Serve. Agent tools/list, resources/list, prompts/list requests hit McpCacheListServer, which reads the cached payload from the store and emits it as DATA + END without forwarding.
• Periodic refresh. After each kind's terminal write (cache.put completion, bodyLen==0 skip-put, or abort/reset), the binding schedules signaler.signalAt(now + ttl) for that kind. On expiry, a new list stream is issued on the same hydrate session and the cache value is overwritten. Aborts during refresh preserve the prior cached entry.
• Lease coordination across workers. A binding-wide lifecycle.lock arbitrates which worker opens the hydrate session (loser polls at 100 ms intervals); a per-kind <kind>.lock arbitrates which worker issues each list call on the wire. Winners release on hydrate complete (lifecycle lease) or after cache.put / abort terminal (per-kind lease). Settle paths split: cache hits funnel through markSettled so only the worker that actually did the list-stream work arms a refresh signal.
• Notification-driven refresh (notifications/{tools,resources,prompts}/list_changed) — deferred per the issue description.

Refactor

McpProxyFactory ballooned absorbing cache support. Extracted into four per-kind-family factories, each implementing BindingHandler and dispatched from McpProxyFactory.factories: Int2ObjectHashMap<BindingHandler>:

• McpProxyLifecycleFactory — KIND_LIFECYCLE.
• McpProxyItemFactory (abstract base + McpProxyToolsCallFactory / McpProxyPromptsGetFactory / McpProxyResourcesReadFactory) — KIND_TOOLS_CALL / KIND_PROMPTS_GET / KIND_RESOURCES_READ.
• McpProxyListFactory (abstract base + McpProxyToolsListFactory / McpProxyPromptsListFactory / McpProxyResourcesListFactory) — KIND_TOOLS_LIST / KIND_PROMPTS_LIST / KIND_RESOURCES_LIST. newStream branches internally on binding.cache != null to pick McpListServer (passthrough) or McpCacheListServer (serve-from-cache).

McpProxyFactory shrinks from ~3631 lines to ~566 lines and now owns only the dispatch map, attach/detach, and the multi-kind hydrate coordinator (McpHydrateSession + McpHydrateListStream) which cannot be cleanly carved per kind.

Spec scripts

Folder	Verifies
cache.hydrate	hydrate session opens with lifecycle
cache.hydrate.tools / .resources / .prompts	each kind populated during hydrate
cache.hydrate.persist	lifecycle stays open after all three list streams close
cache.hydrate.error	downstream abort during hydrate; lifecycle survives
cache.hydrate.auth	hydrate BEGIN carries the guard-issued auth token
cache.serve.initialize	agent observes lifecycle initialised once cache is ready
cache.serve.tools.list / .resources.list / .prompts.list	each list served from cache, no downstream call
cache.refresh.tools / .resources / .prompts	second list call after TTL on same hydrate session
cache.refresh.tools.error	refresh aborts; prior cached entry retained
cache.refresh.tools.contended	two workers, exactly one issues each list call on the wire

ITs

Peer-to-peer (specs/binding-mcp.spec/.../streams/cache/)	Engine-driven (runtime/binding-mcp/...)
ProxyCacheLifecycleIT (5 tests)	McpProxyCacheLifecycleIT (5 tests)
ProxyCacheToolsListIT (5 tests)	McpProxyCacheToolsListIT (4 tests)
ProxyCacheResourcesListIT (3 tests)	McpProxyCacheResourcesListIT (3 tests)
ProxyCachePromptsListIT (3 tests)	McpProxyCachePromptsListIT (3 tests)
—	McpProxyCacheContentionIT (1 test, ENGINE_WORKERS=2)

Every @Ignore from the original test-first scaffold has been removed.

Test-only configs

• proxy.cache.yaml — empty test store, single worker; used by hydrate ITs.
• proxy.cache.refresh.yaml — empty test store with per-kind TTLs; refresh + contention ITs.
• proxy.cache.seeded.yaml — test store pre-seeded with the cached JSON; serve ITs.
• proxy.cache.auth.yaml — test store + engine test guard with credentials; auth IT.
• proxy.cache.toolkit.yaml — multi-route variant for toolkit-flavoured tests.

TestStore enhancement (runtime/engine test sources)

TestStore now owns ConcurrentMap<Long, ConcurrentMap<String, String>> keyed by storeId; workers attached to the same store share entries. options.entries seeds via putIfAbsent so the first attach wins. Previously the entries map was per-handler, so putIfAbsent-based leases were trivially won by every worker — required for McpProxyCacheContentionIT to observe a single winner. Transparent for single-worker scenarios. binding-mcp no longer test-depends on store-memory; all cache ITs run on type: test.

Test plan

• Peer-to-peer ITs (specs/binding-mcp.spec ProxyCache*IT) → 16 pass / 0 fail.
• Engine-driven binding-mcp ITs (runtime/binding-mcp) → 158 pass / 0 fail / 0 skipped.
• specs/engine.spec → 39 pass / 0 fail after the TestStore sharing change.

🤖 Generated with Claude Code

---

Generated by Claude Code

[jfallows]

Remove these type of comments from the ITs.
The scenarios and scripts should be the documentation.

[jfallows]

proxy.cache.toolkit.yaml is sufficient here, even though it has multiple toolkit routes.

[jfallows]

Suggested change

	routes:
	- exit: app1
	exit: app1

[jfallows]

Remove these kinds of comments from the spec scripts.

[jfallows]

This should not be hardcoded in the implementation.
Instead it should use a session id supplier, same as other binding types, and override to fix the value for the ITs that need a deterministic value.

[jfallows]

Suggested change

	private final class HydrateSession
	private final class McpHydrateSession

[jfallows]

I think this initialization logic should belong in the McpBindingConfig constructor, to initialize cache and then this code block can be gated on if (binding.cache != null) instead.

[jfallows]

Suggested change

	private record PendingAwait(
	long originId,
	long routedId,
	long streamId,
	long traceId,
	int signalId)
	{
	}
	private record McpSignalHandle(
	long originId,
	long routedId,
	long streamId,
	long traceId,
	int signalId)
	{
	}

Perhaps have a convenience method to invoke the signaler, passing all the record fields to signaler.signalNow?

[jfallows]

This can move to a method called onAcquireLifecycleLeaseComplete, passed to cache.acquireLifecycleLease.

Note: the traceId can be moved into the method body.

[jfallows]

Avoid multiple returns, instead conditional surrounds remainder of body with inverted condition.

[jfallows]

This approach seems overly complex.
Perhaps it would simplify If we had separate signal handlers for refreshing each different kind, and initially populating the cache is treated as just the first refresh.

Perhaps we can move the signaling to mark complete into the McpListCache instead, and then McpCacheHydrater can simplify to be per kind only.

[jfallows]

This should be a standard doInitialEnd method.

[jfallows]

Move these to top level classes.

[jfallows]

Move these to top level classes.

[jfallows]

Move these to top level classes.

[jfallows]

Suggested change

	public static String hydrateSessionId()
	{
	return "hydrate-1";
	}
	public static String sessionId()
	{
	return "hydrate-1";
	}

[jfallows]

Suggested change

	public static IntPredicate hydrateKindFilter()
	{
	return kind -> kind == KIND_TOOLS_LIST;
	}
	public static IntPredicate hydrateToolsOnly()
	{
	return kind -> kind == KIND_TOOLS_LIST;
	}

[jfallows]

Suggested change

	public static final String MCP_HYDRATE_KIND_FILTER_NAME = "zilla.binding.mcp.hydrate.kind.filter";
	public static final String MCP_HYDRATE_FILTER_NAME = "zilla.binding.mcp.hydrate.filter";

[jfallows]

Suggested change

	assertEquals(MCP_HYDRATE_KIND_FILTER.name(), MCP_HYDRATE_KIND_FILTER_NAME);
	assertEquals(MCP_HYDRATE_FILTER.name(), MCP_HYDRATE_FILTER_NAME);

[jfallows]

Use multi-line yaml syntax to make this more readable.

[jfallows]

See feedback.

[jfallows]

Can this be made final?

[jfallows]

Let's avoid passing kind to super and have abstract overrides for each part that needs kind specific awareness.

[jfallows]

Let's change these to remove the _MS suffix and make their type Duration instead, similar to MAX_AGE above.

[jfallows]

Suggested change

	public static final String MCP_LEASE_TTL_MS_NAME = "zilla.binding.mcp.lease.ttl.ms";
	public static final String MCP_LEASE_RETRY_MS_NAME = "zilla.binding.mcp.lease.retry.ms";
	public static final String MCP_LEASE_TTL_NAME = "zilla.binding.mcp.lease.ttl";
	public static final String MCP_LEASE_RETRY_NAME = "zilla.binding.mcp.lease.retry";

[jfallows]

Suggested change

	assertEquals(MCP_LEASE_TTL_MS.name(), MCP_LEASE_TTL_MS_NAME);
	assertEquals(MCP_LEASE_RETRY_MS.name(), MCP_LEASE_RETRY_MS_NAME);
	assertEquals(MCP_LEASE_TTL.name(), MCP_LEASE_TTL_NAME);
	assertEquals(MCP_LEASE_RETRY.name(), MCP_LEASE_RETRY_NAME);

[jfallows]

Suggested change

	tools: |-
	{"tools":[{"name": "get_weather","title": "Weather Information Provider","description": "Get current weather information for a location","inputSchema": {"type": "object","properties": {"location": {"type": "string","description": "City name or zip code"}},"required": ["location"]},"icons": [{"src": "https://example.com/weather-icon.png","mimeType": "image/png","sizes": ["48x48"]}],"execution": {"taskSupport": "optional"}}]}
	tools: |-
	{
	"tools":
	[
	{
	"name": "get_weather",
	"title": "Weather Information Provider",
	"description": "Get current weather information for a location",
	"inputSchema":
	{
	"type": "object",
	"properties":
	{
	"location":
	{
	"type": "string",
	"description": "City name or zip code"
	}
	},
	"required":
	[
	"location"
	]
	},
	"icons":
	[
	{
	"src": "https://example.com/weather-icon.png",
	"mimeType": "image/png",
	"sizes": ["48x48"]
	}
	],
	"execution":
	{
	"taskSupport": "optional"
	}
	}
	]
	}

Perhaps we can use this if it folds to JSON with minimal whitespace.

[jfallows]

Suggest similar formatting to above feedback for these if it works out for tools.

[jfallows]

If the McpLifecycleCache is 1-1 with the McpBindingConfig, then does it make sense to move the cache guard and cache credentials onto the cache, as guard and credentials?
If we scope things correctly, then it seems like the McpProxyCacheHydrater would only need the McpLifecycleCache and not the binding, agree?

[jfallows]

Suggest updating the acquireXxx methods to take duration, then convert toMillis() as needed in the acquireXxx implementation.

[jfallows]

Suggested change

	signaler.signalAt(currentTimeMillis() + leaseRetry.toMillis(), SIGNAL_INITIATE_LIFECYCLE,
	this::onInitiateLifecycle);
	signaler.signalAt(Instant.now().plus(leaseRetry).toEpochMillis(), SIGNAL_INITIATE_LIFECYCLE,
	this::onInitiateLifecycle);

Let's also add an overload of Signaler.signalAt that takes an Instant instead of long, and fills in default method via Instant.toEpochMillis().

[jfallows]

This method should not be called from the constuctor.
The creator should explicitly call doLifecycleBegin post construction at the constructor call site.

[jfallows]

Suggested change

	private void onInitialAcquireLeaseComplete(
	private void onInitialAcquireComplete(

[jfallows]

Suggested change

	private void onRefreshAcquireLeaseComplete(
	private void onRefreshAcquireComplete(

[jfallows]

Avoid doing this behavioral work in the constructor.
Instead have a doListHydrateBegin(...) and a doListHydrateEnd(...) and call them from the call site of the constructor.

[jfallows]

Need a proper reaction to onListHydrateAbort -> doListHydrateAbort, and onListHydrateReset -> doListHydrateReset with appropriate streamId, ie. initialId or replyId, and gated by McpState.initial/replyClosed.