Replies: 2 comments 10 replies
|
I think the core contract needs one more separation: a successful search should not immediately make every matched tool schema visible. The model needs a lightweight way to understand the available tool space before searching, and it should get another opportunity to confirm the candidates before Maka sends full schemas. A simple flow is: The states are distinct:
For provider step Neither search nor load changes This addresses two practical problems:
The initial discovery surface can stay small: expose only Under this model, item 6 can still retire economy/full mode,
Search indexes and matching results may be cached, provided cached candidates are revalidated against the current bindings. Provider or prompt caching is a downstream transport optimization and should not be part of the lazy-loading contract. The tradeoff is that the initial group/name index still grows with the number of bound tools, but it is much smaller than publishing every description and input schema, while avoiding blind trial-and-error discovery. AI assistance disclosure: Maka helped analyze the state transitions and draft this comment; I reviewed the conclusion and posted it. |
|
Thanks. I agree that separating candidate discovery from full-schema visibility is a coherent design. I am not convinced that the extra confirmation step belongs in the first slice. The product goal here is to reduce tool tax. I checked a local Maka Dev runtime snapshot from Aug 7 to Aug 24: 63 sessions, 23,450 messages, and 8,710 tool invocations. The current deferred catalog is four groups and 15 tools: Rive, Browser, Computer Use, and Agent. That is large enough for group loading to become coarse, but not yet evidence that Maka needs a three-stage discovery protocol. The comparison with other agents is mixed. Codex uses a single There are also valid multi-stage designs. Hermes uses Kimi Code is also not the architecture I would use as a reference for Maka. In our Terminal-Bench 2.1 comparison, Maka scored 62/89 versus Kimi Code at 53/89. Among tasks that finished before the deadline, the rates were 95.1% versus 85.7%. The report also documents important harness differences, so I would not attribute the whole gap to For the first slice, I would use one Maka-owned discovery seam:
For providers without native deferred search, the Runtime can project the selected schemas into the next request. For providers that support native search later, that can become an optimization behind the same contract. I would not add I would measure result count, schema bytes, time to the first successful tool call, extra round trips, repeated searches, and invalid selections. If those measurements show that direct search-to-activation exposes too many schemas or causes frequent wrong selections, an explicit confirmation boundary can be added later. AI assistance disclosure: Codex helped me inspect the Maka runtime data and related agent implementations and draft this comment; I reviewed and approved the final position. 中文谢谢。我同意把候选发现和完整 schema 暴露分开在技术上是合理的,但我不认为额外的确认步骤应该放进第一阶段。 这里的产品目标是减少工具税。 我检查了 8 月 7 日到 8 月 24 日的本地 Maka Dev 运行快照:63 个 session、23,450 条消息和 8,710 次工具调用。 当前 deferred catalog 是 4 组、15 个工具:Rive、Browser、Computer Use 和 Agent。这个规模已经足以让 group loading 开始变粗,但还不足以证明 Maka 需要三阶段 discovery 协议。 其他 Agent 的做法并不统一。Codex 使用单一的 当然也有合理的多阶段设计。Hermes 使用 Kimi Code 也不是我会拿来作为 Maka 架构参考的实现。在我们的 Terminal-Bench 2.1 对比中,Maka 是 62/89,Kimi Code 是 53/89;在 deadline 前完成的任务中,两者分别是 95.1% 和 85.7%。报告也记录了重要的 harness 差异,所以我不会把全部差距归因于 第一阶段我会采用一个 Maka 自己拥有的 discovery 入口:
对不支持原生 deferred search 的 provider,Runtime 可以把选中的 schema 投影到下一次请求。以后支持原生 search 的 provider 可以在同一契约后面做优化。第一阶段我不会增加 我会测量搜索结果数量、schema 字节数、第一次成功工具调用的延迟、额外往返次数、重复搜索和非法选择。如果数据证明直接 search-to-activation 暴露了过多 schema,或者经常选错,再增加显式确认边界。 AI assistance disclosure:Codex 协助我检查 Maka 运行数据和相关 Agent 实现,并起草了这条评论;最终立场由我审阅并确认。 |
Uh oh!
There was an error while loading. Please reload this page.
Context
Maka currently advertises a large live tool surface and controls it through several overlapping mechanisms: economy mode, named
load_toolsgroups,MAKA_DISABLE_DEFERRED_TOOLS, historical ledger aliases, and exact hosted-execution tool ceilings. This makes the model-facing contract difficult to explain and leaves more than one source of truth for tool identity and availability.Issue #1382 proposed a provider-native-first Tool Search design. The implementation audit suggests a simpler provider-independent contract that works across every model backend and does not persist model-specific discovery state.
Proposed direction
Bash,Read,ArchiveRead,Write,Edit,Glob,Grep,WebFetch,AskUserQuestion, andStopBackgroundTask.tool_searchtool. A successful search selects bounded matching tools and makes their schemas available on the next model step.WriteStdin. Context-required tools may still be injected directly when the corresponding runtime state requires them.MAKA_DISABLE_DEFERRED_TOOLS, namedload_toolsgroups, and historical model-facing alias recovery oncetool_searchcovers their use cases.boundToolsor a hosted tool profile, discovery must not escape that ceiling.Skill/SkillSearchare intentionally excluded pending a separate discussion about skill discovery and host injection. Goal tools are also discussed separately because their public shape can be simplified before deciding how they are exposed.Questions to settle
If this direction holds, #1382 can track the implementation but should be updated away from its provider-native-first and cross-turn persistence assumptions. Follow-up implementation issues should be split by independently revertible intent rather than by internal file.
AI assistance disclosure: Maka helped audit the current implementation and draft this proposal. I reviewed the scope and proposed direction.
All reactions