Skip to content

feat: add optional you.com search integration - #5

Open
mouse-value-add wants to merge 1 commit into
lajosdeme:mainfrom
mouse-value-add:feat/youcom-search-integration
Open

feat: add optional you.com search integration#5
mouse-value-add wants to merge 1 commit into
lajosdeme:mainfrom
mouse-value-add:feat/youcom-search-integration

Conversation

@mouse-value-add

Copy link
Copy Markdown

Adds You.com as a third search provider behind the existing Provider interface, alongside Brave and Tavily. It only activates when search.provider is set to youcom — nothing changes for existing configurations.

Why this fits mole

Two properties of the You.com Search API map directly onto things the search package already cares about:

  • Page content per result. The API accepts extraction_mode: "full_page" and returns each result with a contents.markdown field carrying the page text. That feeds the same fetch-skipping path Tavily uses — Result.Content + HasUsableContent() — so the WebActor can ground claims without a subsequent fetch (§10.4). The provider is chosen for exactly this in IncludeContent mode.
  • Server-side domain filters. include_domains / exclude_domains are request parameters, so both options travel to the provider instead of Brave's query-operator / client-side workaround.

Auth is X-API-Key, which is already in the cassette redaction list, so keys stay out of cassettes without any change to the recorder.

What changed

  • internal/tools/search/youcom.go — the provider. POST to https://ydc-index.io/v1/search, maps results.web (url, title, description, snippets, page_age, contents.markdown), drops empty-URL rows, leaves unparseable dates nil rather than guessing (same rationale as parseBraveAge). Default cost 5,000 micros ($5/1000 queries, entry tier); the full-page extraction surcharge is documented in a comment rather than silently folded into the default.
  • internal/tools/search/search.goKindYoucom in the registry, newYoucom in the factory, package doc updated.
  • internal/config/config.gosearch.youcom-key field (secret), ActiveKey() case, env override MOLE_YOUCOM_API_KEY / YDC_API_KEY, provider validation accepts youcom.
  • cmd/mole/main.gomole doctor reports youcom with the same "returns page content, skips fetches" note Tavily gets, since it changes how many fetches a session makes.
  • cmd/mole/research.go — the "no search provider selected" hint lists youcom.
  • README.md — config example.
  • Tests follow the existing fixture pattern; the Kinds()-iterating tests (error classification, selectability) pick the new provider up automatically.

Setup

mole config set search.provider youcom
mole config set search.youcom-key <key from you.com/platform/api-keys>

Env-only works too: MOLE_SEARCH_PROVIDER=youcom + YDC_API_KEY.

Validation

  • go build ./..., go vet ./... — clean
  • go test ./... — 33 packages pass (new: TestYoucomParsesResults, TestYoucomDomainFiltersAreSentServerSide, TestYoucomSnippetsFallbackForDescription; TestErrorsAreClassifiedForTheRetryPolicy and TestProviderIsSelectable cover youcom via Kinds())
  • go test -race ./internal/tools/search/ ./internal/config/ ./cmd/mole/ — clean
  • Response shape verified against the documented API shape before writing the parser

Happy to adjust the shape if you'd rather see the extraction surcharge priced into queryCost(), or the provider named differently.

Adds youcom as a third search provider behind the existing Provider
interface. You.com's Search API can return full page content with each
result (extraction_mode=full_page), which feeds the same fetch-skipping
path Tavily uses, and it filters domains server-side, which Brave
cannot. Activates only when search.provider is set to youcom.
@mouse-value-add

Copy link
Copy Markdown
Author

Brought this back up to date with the current main — the branch was clean against it (no conflicts). CI on the fork is green. The PR should now show as mergeable; happy to make any other adjustments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant