✨ feat(agent-memory): preserve contrastive insights in skill consolidation

[JadeeeZh]

Problem

AGENT_SKILL_SUCCESS_EXTRACT_PROMPT consolidates a cluster's majority pattern. When most cases in a cluster succeeded via the standard checklist, the minority lessons carried by failed-then-corrected cases (root causes the standard checks missed) get averaged away — even when every individual AgentCase.key_insight states them correctly.

We hit this on a τ²-bench telecom support benchmark: 74 support calls consolidated into skills whose steps dropped the decisive correction entirely, and one skill even codified the trap as a decision branch — literally If roaming enabled → proceed, when the failed cases showed that check passing while the problem persisted. Case layer: lesson present (150 mentions). Skill layer: gone.

Change (prompt-only, domain-agnostic)

• New GOOD-skill criterion: preserve hard-won corrections that distinguish look-alike situations.
• New "Contrastive insight preservation" hard rules:
  • failure-derived root-cause checks must appear as Steps/Decision branches (not only Pitfalls) and survive later condensation;
  • no if <check> passes → proceed branch when any case shows <check> passing while the problem persisted;
  • look-alike variants require an explicit discriminator, not just both fixes listed;
  • capabilities governed by multiple independent controls are verified per control.
• Pitfall cap 4 → 6, and failure-derived pitfalls can no longer be evicted by generic advice.
• Update-op guidance: contrastive checks and failure-derived pitfalls are load-bearing — preserve or strengthen, never summarize away.

No API, schema, or output-format changes; the JSON operation contract and placeholders are untouched. CHANGELOG [Unreleased] updated.

Validation

A/B on τ²-bench telecom (same 74 recorded train calls replayed through the pipeline, same models/seeds, only this prompt differing), 40 held-out tasks vs a no-memory baseline:

	stock prompt	this prompt
Held-out resolution	50% (20/40)	62% (25/40)
Paired vs baseline	2 wins / 2 losses	6 wins / 1 loss
vs stock-prompt arm (same tasks)	—	5 wins / 0 losses

5 of the 6 wins are exactly the failure family the stock prompt had flattened; the consolidated skills retain the correction after all 74 incremental updates. Honest caveats: n=40, discordant pairs 6:1 (one-sided binomial p≈0.063) — directionally strong, below conventional significance; the benchmark is one domain.

Tests

• uv run pytest packages/everalgo-agent-memory/tests — 275 passed
• uv run ruff check / ruff format --check — clean
• The only test touching this constant monkeypatches it, so no content assertions were affected.

🤖 Generated with Claude Code