Add /benchmarks page: LoCoMo & LongMemEval results vs. mem0 - #226
Closed
jy7lsna wants to merge 2 commits into
Closed
Add /benchmarks page: LoCoMo & LongMemEval results vs. mem0#226jy7lsna wants to merge 2 commits into
jy7lsna wants to merge 2 commits into
Conversation
Contributor
|
@jy7lsna is attempting to deploy a commit to the Statewave Team on Vercel. A member of the Team first needs to authorize it. |
jy7lsna
force-pushed
the
feature/benchmarks-page
branch
from
August 3, 2026 14:00
7a26a87 to
49b63db
Compare
Adds a detailed, reproducible benchmark writeup (scoreboard, methodology, run-it steps, scope, governance comparison, FAQ) run on mem0's own eval harness, plus the categorical chart palette and icon-only CodeCopyButton variant it needed.
jy7lsna
force-pushed
the
feature/benchmarks-page
branch
from
August 7, 2026 06:26
49b63db to
d6aa40d
Compare
… the closing CTA The "Diff the fork against upstream" button compared statewave-memory-benchmarks against itself. Extract UPSTREAM_DIFF_URL so it compares against mem0ai/memory-benchmarks, the repo it was forked from. Also brings the closing CTA in line with the site's cta-card treatment.
smaramwbc
pushed a commit
that referenced
this pull request
Aug 10, 2026
Four page and content additions, landed together: - /benchmarks — LoCoMo and LongMemEval results for Statewave, mem0 cloud and mem0 OSS, run in-harness with a shared gpt-4o judge; numbers trace to the statewave-memory-benchmarks repo. - /vs/mem0 — comparison page: deterministic, token-bounded context assembly with policy enforcement and receipts vs. ranked similarity retrieval. - /use-cases/grounded-shop-assistant — use-case page for the grounded shop assistant reference build (links smaramwbc/statewave-grounded-shop-assistant). - Custom header and OG images for all five blog posts. Combined from PRs #226, #228, #235 and #239 (see those PRs for the full per-change history and discussion).
Owner
|
Shipped 🎉 — this landed on main via #237 (squashed as 5d76012) together with #228, #235 and #239. The /benchmarks page is live with your numbers exactly as written — we verified every figure traces to statewave-memory-benchmarks before merging. Thanks for the careful, source-backed work; closing since the content is merged. |
18 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds a new
/benchmarkspage presenting Statewave's LoCoMo and LongMemEval results head-to-head against mem0 (cloud and OSS), run on mem0's own eval harness with gpt-4o as a shared answerer/judge across all three backends. The page includes an interactive scoreboard, methodology writeup, copy-pasteable reproduction steps per backend, an explicit scope section (what is and isn't claimed), a governance comparison table, and an FAQ. Wired into routing, nav, footer, and the SEO/sitemap route table so it's discoverable and indexable.Related Issue
Closes #
Type of Change
Changes Made
src/pages/BenchmarksPage.tsx: scoreboard (with axis zoom/full toggle and series isolation), methodology pipeline diagram, per-backend reproduce steps, scope/claims section, governance comparison matrix, and FAQ/benchmarksintosrc/App.tsx(route),src/components/Navbar.tsx(nav link),src/components/Footer.tsx(footer link), andsrc/lib/seo-meta.ts(title/description/og:type/sitemap entry)--series-statewave/--series-mem0-cloud/--series-mem0-osscategorical chart color tokens tosrc/index.cssfor light and dark themes, validated for CVD contrast, plus a.section-glow--softutility for sections carrying saturated chart huesiconOnlyvariant tosrc/components/CodeCopyButton.tsxfor dense terminal-panel headersTesting
Test Commands Run
Checklist
Screenshots / Recordings









