-
Notifications
You must be signed in to change notification settings - Fork 0
Examples
Klein Panic edited this page Apr 4, 2026
·
1 revision
Real-world query examples demonstrating Memory-Spark pipeline behavior, tuning, and expected results.
Query: "What is Klein's preferred timezone?"
Agent: meta
Pipeline trace:
Stage 5 (Embed): 42ms
Stage 6 (Search):
agent_memory: 4 results (USER.md, MEMORY.md, memory/2026-03-01.md, memory/2026-02-15.md)
shared_rules: 1 result (shared-user-profile.md)
Stage 7 (RRF): 5 unique candidates
Stage 9 (Weighting): USER.md chunk boosted 1.3x
Stage 12 (Gate): spread=0.22 > 0.08 → SKIP reranker
Stage 13 (MMR): 3 diverse results
Top result (score=1.0):
path: "USER.md"
snippet: "- **Timezone:** America/New_York"
source: memory
Query: "How do I configure the BlueBubbles MCP server?"
Agent: dev
Pipeline trace:
Stage 6 (Search):
agent_memory: 6 results (TOOLS.md with 1.3x boost, MEMORY.md with 1.4x boost)
shared_knowledge: 3 results (BlueBubbles skill docs)
Stage 9 (Weighting): TOOLS.md chunk → 1.3x, capture → 1.5x
Stage 12 (Gate): spread=0.06 → RUN reranker (ambiguous)
Stage 12 (Reranker): 119ms, logitSpread=2.8
Stage 13 (MMR): removed 2 near-duplicates (TOOLS.md and MEMORY.md had overlapping BlueBubbles info)
Top result (score=0.94):
path: "TOOLS.md"
snippet: "## BlueBubbles MCP (v2.0)... mcporter call bluebubbles.<tool>..."
Query: "Have we seen this error before: Arrow schema mismatch on mergeInsert?"
Agent: dev
Pipeline trace:
Stage 6 (Search):
agent_mistakes: 2 results (dev's own MISTAKES.md entries, 1.6x boost)
shared_mistakes: 1 result (meta had the same issue, 1.6x boost)
agent_memory: 3 results (MEMORY.md references to LanceDB issues)
Stage 9 (Weighting):
agent_mistakes chunk: score 0.82 × 1.6 = 1.312
shared_mistakes chunk: score 0.78 × 1.6 = 1.248
agent_memory chunk: score 0.85 × 1.0 = 0.85
Stage 11 (Normalize): top=1.0 (1.312 → 1.0, 1.248 → 0.951, 0.85 → 0.648)
Result: Mistakes are surfaced first despite lower raw cosine scores, because:
1. Source weight 1.6x for mistakes pools
2. After normalization, mistake chunks rank above plain memory chunks
Query: "What format should I use for daily notes?"
Agent: school
Pipeline trace:
Stage 6 (Search):
agent_memory: 2 results
shared_rules: 1 result (from shared-preferences.json, always-inject pool)
shared_knowledge: 1 result (from Documentation/Agents/)
shared_rules chunks are always searched regardless of relevance score
(lowThreshold = max(minScore × 0.7, 0.05) = max(0.525, 0.05) = 0.525)
# Reference pools are NEVER auto-injected. Must use explicit memory_search:
memory_search({
query: "OpenClaw cron syntax",
maxResults: 5
})
# The auto-recall hook searches these pools:
# ✅ agent_memory, agent_tools, agent_mistakes
# ✅ shared_knowledge, shared_mistakes, shared_rules
# ❌ reference_library (tool-call only)
# ❌ reference_code (tool-call only)
# To search reference pools, the agent must explicitly call memory_search
# which searches across ALL pools including reference
Query: "What did we work on regarding the Mika LoRA?"
Agent: meta
Temporal decay applied (floor=0.8, rate=0.03):
memory/2026-04-03.md (age: 0 days)
raw_score=0.88, decay=1.00, final=0.88
memory/2026-03-28.md (age: 6 days)
raw_score=0.91, decay=0.96, final=0.874
memory/2026-03-01.md (age: 33 days)
raw_score=0.93, decay=0.88, final=0.818
MEMORY.md (evergreen, no decay)
raw_score=0.85, decay=1.00, final=0.85
Result ordering: today's note > MEMORY.md > last week > last month
Even though the 33-day-old note had the highest raw similarity (0.93),
temporal decay pushed it below the 0-day and evergreen entries.
# These paths are immune to temporal decay:
MEMORY.md → evergreen (root memory file)
memory.md → evergreen (alt root)
memory/infrastructure.md → evergreen (non-dated topic file)
memory/coding-standards.md → evergreen (non-dated topic file)
# These paths DO decay:
memory/2026-01-15.md → date from filename (age: 78 days → 0.81x)
memory/2026-04-01.md → date from filename (age: 2 days → 0.99x)
# Files without dates in filename:
memory/session-notes.md → mtime from filesystem
Query: "How does the gateway restart process work?"
Agent: meta
Without MMR (raw reranker output):
1. AGENTS.md:350-380 (restart approval handler) score=0.95
2. AGENTS.md:380-410 (oc-restart workflow) score=0.93 ← near-duplicate!
3. AGENTS.md:410-440 (config change workflow) score=0.91 ← near-duplicate!
4. MEMORY.md:120-140 (restart troubleshooting note) score=0.88
5. memory/2026-03-09.md (restart incident) score=0.85
With MMR (λ=0.9):
1. AGENTS.md:350-380 (restart approval handler) score=0.95 ← kept (first pick)
2. MEMORY.md:120-140 (restart troubleshooting note) score=0.88 ← promoted (diverse)
3. memory/2026-03-09.md (restart incident) score=0.85 ← promoted (diverse)
4. AGENTS.md:380-410 (oc-restart workflow) score=0.93 ← demoted (similar to #1)
5. AGENTS.md:410-440 (config change workflow) score=0.91 ← demoted (similar to #1)
MMR removed 2 near-duplicate AGENTS.md chunks from top positions,
promoting diverse content from MEMORY.md and daily notes.
# When configured with mmrLambda: 0.5 (more diversity):
Query: "Tell me about our infrastructure"
Results would include one chunk from each topic area:
1. Hardware inventory (server specs)
2. Network configuration (Tailscale, VPN)
3. Service map (ports, processes)
4. Security policies (nftables, SSH)
5. Monitoring setup (immune agent)
Instead of 5 chunks all about server hardware specs.
Pipeline log output:
[recall] MMR: configured λ=0.9 | adaptive would be λ=0.95 (spread=0.350, tier=wide)
[recall] MMR: configured λ=0.9 | adaptive would be λ=0.85 (spread=0.150, tier=medium)
[recall] MMR: configured λ=0.9 | adaptive would be λ=0.70 (spread=0.050, tier=tight)
Tiers:
Wide spread (>0.3): λ=0.95 — ranking is confident, trust relevance
Medium (0.1-0.3): λ=0.85 — moderate diversity helps
Tight cluster (<0.1): λ=0.70 — diversity needed to break ties
Query: "What is the Spark embedding port?"
Vector scores (top 5):
#1: 0.94 (MEMORY.md: "Embedding: port 18091")
#2: 0.85 (Configuration.md: "embed.port: 18091")
#3: 0.79 (TOOLS.md: "Spark services")
#4: 0.71 (memory/infra.md: "DGX services")
#5: 0.65 (memory/2026-03.md: "Spark setup")
Spread = 0.94 - 0.65 = 0.29 > 0.08
→ GATE SKIP: "hard-gate-high: spread=0.2900 > 0.08 (vector confident)"
→ Return vector order directly, save 119ms reranker call
Query: "agent configuration"
Vector scores (top 5):
#1: 0.72 (AGENTS.md section A)
#2: 0.71 (AGENTS.md section B)
#3: 0.71 (AGENTS.md section C)
#4: 0.70 (AGENTS.md section D)
#5: 0.70 (AGENTS.md section E)
Spread = 0.72 - 0.70 = 0.02 < 0.02
→ GATE SKIP: "hard-gate-low: spread=0.0200 < 0.02 (tied set)"
→ All candidates are equally relevant; reranker would just randomly reshuffle
Query: "How do I handle SSH key rotation on the server?"
Vector scores (top 5):
#1: 0.83 (memory/ssh-setup.md)
#2: 0.80 (TOOLS.md SSH section)
#3: 0.79 (memory/2026-02.md server maintenance)
#4: 0.77 (shared-knowledge/security.md)
#5: 0.76 (MISTAKES.md SSH error)
Spread = 0.83 - 0.76 = 0.07
→ 0.02 ≤ 0.07 ≤ 0.08 → RUN reranker
→ Reranker re-scores with cross-encoder (query-document pairs)
→ May promote MISTAKES.md SSH error to #1 (most actionable for the query)
# With rerankerGate: "soft"
Query: "deployment checklist"
Spread = 0.12 → above lowThreshold (0.02), below threshold (0.08)?
Actually spread=0.12 > threshold(0.08):
multiplier = 1.0 (full vector trust)
If spread was 0.05 (in [0.02, 0.08]):
t = (0.05 - 0.02) / (0.08 - 0.02) = 0.5
multiplier = 0.5 + 0.5 × 0.5 = 0.75
→ Vector weight in RRF scaled to 0.75 (reranker gets more influence)
Query: "spark models"
Without HyDE:
Raw query embedding matches generic "models" content
Top result: unrelated ML model discussion (score 0.72)
With HyDE:
Hypothetical doc generated:
"The DGX Spark node runs several NVIDIA models for different tasks.
The embedding model is llama-embed-nemotron-8b providing 4096-dimensional
vectors. The reranker model is llama-nemotron-rerank-1b-v2 for cross-encoder
scoring. The main LLM is Nemotron-Super-120B for generation tasks."
Hypothetical embedded as DOCUMENT (no instruction prefix)
Top result: Infrastructure doc about Spark services (score 0.91)
Query: "what was that config change we made?"
[recall] HyDE: generating hypothetical document for query="what was that config change we made?"
[recall] HyDE: generation returned empty/null — falling back to raw query embedding
(LLM returned empty because the query is too vague for a useful hypothetical)
→ Falls back to raw query embedding silently
→ Search still works, just with less precision
Query: "What are Klein's coding preferences?"
Recent messages already contain (via LCM summary):
"Klein prefers concise code, TypeScript, and functional patterns.
He values minimal dependencies and clean error handling."
Memory-Spark recalls chunk from MEMORY.md:
"Klein's coding style: concise TypeScript, functional patterns,
minimal dependencies, explicit error handling."
Token overlap check:
chunkTokens: {concise, typescript, functional, patterns, minimal, dependencies, error, handling}
contextTokens: {concise, code, typescript, functional, patterns, minimal, dependencies, clean, error, handling}
overlap = 8/8 = 1.0 > threshold(0.4)
→ DROPPED (LCM already provides this information)
Query: "What is the restart approval process?"
Search finds child chunk (200 tokens, precise match):
id: "child-abc123"
parent_id: "parent-xyz789"
text: "oc-restart request validates config, posts Discord card..."
score: 0.92
Pipeline expands to parent chunk (2000 tokens, full context):
id: "parent-xyz789"
text: "## Restart Approval Handler (MANDATORY)\n\n**The single restart interface..."
(includes complete handler documentation)
Injected context uses parent's full text (2000 tokens)
instead of child's snippet (200 tokens) — much more useful for the agent.
If 3 children from the same parent matched, only the highest-scoring
one's parent text is included (dedup by expanded parent_id).
# The reranker (Nemotron-Rerank-1B) was trained on Q&A pairs.
# Declarative statements produce compressed scores.
Input: "0-dimensional biomaterials show inductive properties"
Output: "Is it true that 0-dimensional biomaterials show inductive properties?"
Effect: Score spread increases from ~0.02 to ~0.77
Input: "What model does Spark use for embeddings?"
Output: "What model does Spark use for embeddings?" (already a question, no change)
Input: "Klein prefers TypeScript."
Output: "Is it true that Klein prefers TypeScript?"