feat(research): deep_research tool β autonomous multi-step web research (#293)#294
Merged
Conversation
β¦#293) Plan β search β read (plain HTTP fetch) β iterate on gaps β synthesize a structured, cited report. Sub-calls run on a configurable fast model; synthesis defaults to the main agent model. Hard token budget, depth and max-sources ceilings configurable in the Tools tab; /stop cancels between phases; progress updates stream to the originating chat; report is published as a themed web artifact when the workspace is enabled.
β¦ion/permissions pages
- expanduser on the artifact write path so a '~/β¦' workspace serves what was written (matches artifacts.serving_base) - only URLs actually read are marked seen β candidates dropped by the max_sources cap stay eligible for later cycles - per-source fetch+extract run as one independent chain (no cross-source barrier stalling extraction on the slowest fetch) - extraction failures are logged instead of silently dropped - '##' report sections render as the styled h2 (was h3, inverting the artifact's visual hierarchy) - deep_research honours the per-agent web_search scope (hidden from the offer and refused in the handler) - clear upfront error when the configured sub-call/synthesis provider has no API key (instead of an opaque auth failure mid-run) - Tools tab: one get_many read + defaults sourced from the pydantic model; provider dropdowns reuse the shared LLM_PROVIDERS catalog - dead in_list bookkeeping removed from the report renderer; docstring/UI note that background subagent runs cancel from Jobs, not /stop
- dedupe set stores the normalized (stripped) URL, matching the check - provider-key guard skips the synthesis provider when it isn't actually used (blank synthesis model = main agent LLM) and accepts base_url-only keyless sidecar setups
Owner
Author
|
Self-review done (two passes). Fixed: artifact path expanduser, seen-URL starvation across cycles, per-agent web_search scope bypass, missing-provider-key upfront error (base_url-only sidecars still allowed), silent extraction failures, report heading hierarchy, fetch/extract pipelining, Tools-tab single get_many + defaults sourced from the pydantic model, shared provider dropdown catalog. 1080 tests green, ruff clean. |
Owner
Author
|
Follow-up from a live run: the model's reply omitted the artifact links. The handler now posts the report link straight to the chat (same path as the progress updates) instead of relying on the model to echo it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #293.
What
A new
deep_researchnative tool: plan β search β read β iterate β synthesize, as a deterministic pipeline (core/deep_research.py) rather than a free-form agent loop.web_searchtool (SearXNG or Tavily), URLs deduped across cycleshttpx, already a dep) + stdlib HTMLβtext; no Playwright spin-up. A failed fetch falls back to the search snippetdepthcycles: a gap-analysis call produces follow-up queries, or stops early[n]citationsDecisions (per the issue discussion)
deepseek/deepseek-v4-flash); synthesis defaults to the main agent model (honouring per-agent LLM overrides via_agent_llm), with an optional dedicated synthesis provider/model. All in the Tools tab.artifacts/research-<slug>-<hex>/index.html, one shared look and feel across runs) and the tool instructs the model to reply briefly with takeaways + link. Without a servable workspace, the full Markdown report rides back in the tool result for the model to relay./stop(and the Stop button) cancels it. Inside a background subagent,task.cancel()propagates through the awaits as usual.token_budget(default 300k, counted from real usage), plusdepthandmax_sourcesceilings the model may dial down but never exceed (resolve_cap, same semantics as subagents)._tool_web_search, so SearXNG and Tavily both work.π Researchingβ¦,π Reading N sourcesβ¦,π³οΈ Following up on gapsβ¦,βοΈ Synthesizingβ¦) are sent to the originating chat.spawn_subagent(background=true)with a research task gives a non-blocking run for free.Wiring
DeepResearchToolConfigundertools.deep_research(hot-applied via the existing config PATCH path)GATEABLE_TOOLS; default permissionALWAYS(read-only; budgets cap the cost)admin-ui.mdx,configuration.mdx,permissions.mdxTests
12 new tests in
tests/test_deep_research.py(pipeline happy path, gap iteration + URL dedup, irrelevant-source drop, token-budget early stop, cancellation, max-sources cap, snippet fallback, HTML/text/slug/report-rendering helpers, feature gating). Full suite: 1079 passed.