feat: hybrid search with BM25 + Embedding + RRF fusion

[effortprogrammer]

Problem

BM25 lexical search can't distinguish 'search local files' from 'search GitHub' — tools like grep_app:searchGitHub get selected when the user intends local code search.

Solution

Hybrid search combining BM25 (lexical) + embedding (semantic) via Reciprocal Rank Fusion.

New files

• embedder.ts — Embedder interface with embedSync() for cached sync access
• onnx-embedder.ts — OnnxEmbedder using optional onnxruntime-node, WordPiece tokenizer, mean pooling + L2 normalize
• embedding.ts — EmbeddingSearchEngine pre-computes tool vectors, sync cosine similarity at query time
• hybrid.ts — HybridSearchEngine + RRFStrategy (k=60) combines multiple engines

Modified files

• factory.ts — opt-in via RouterCoreOptions.hybrid + embedder
• index.ts — exports new modules

Design decisions

• SearchEngine.query() remains synchronous — embeddings pre-computed at catalog build, only cosine similarity at query time
• onnxruntime-node is optional — dynamic import, graceful fallback
• Model-agnostic Embedder interface — swap bge-small-en-v1.5 for any model
• RRF fusion is rank-based, not score-based — robust to different score scales between engines

Research

Based on: ToolRet, Tool-DE, RAG-MCP, ScaleCall papers showing hybrid retrieval significantly outperforms BM25-only for tool selection.

Usage

import { createRouterCore, OnnxEmbedder } from 'mcpflow-router';

const embedder = new OnnxEmbedder({
  modelPath: './models/bge-small-en-v1.5/model.onnx',
  tokenizerPath: './models/bge-small-en-v1.5/tokenizer.json',
});

const core = createRouterCore({
  hybrid: true,
  embedder,
});