You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Parent tracker for the bench-coverage gaps surfaced during review of PR #1228 (feature/stable-bench — bench foundation). PR #1228 ships the chassis (fluree-bench-support), 7 new hot-path benches, and the bench-gate CI job. The reviewer flagged eight perf-regression-prone hot paths that PR #1228 doesn't cover — bundled here into 6 sub-issues based on which gaps are best landed together as a single PR.
These are all real coverage gaps (each has a memory ID documenting a recent or known perf-sensitive area). The gate the foundation PR lands today is "smoke + reconcile only" — it catches benches that no longer compile or panic, but does not yet do regression comparison. That's bench-nightly's job once we have committed baselines. The coverage gaps below are independent of bench-nightly: each is "we should have a bench at all for this hot path."
[bench] Lex-sorted-string ORDER BY fast-path regression bench #1230 — Lex-sorted-string ORDER BY fast-path regression bench. 200× speedup that depends on a specific invariant (lex_sorted_string_ids flag preservation through bulk import + first post-import write). Silent regression vector; would slow production ORDER BY queries on freshly imported datasets.
[bench] Filter PreparedBoolExpression cache regression bench #1232 — Filter PreparedBoolExpression cache regression bench. Per-row cacheability analysis is the easy thing to accidentally re-run; the regression vector is silent and filter-heavy queries are common in production.
[bench] Parser micro-benches (Turtle, JSON-LD, SPARQL) #1234 — Parser micro-benches (Turtle, JSON-LD, SPARQL). Parsing is currently exercised only through import_bulk and transact_commit; a parser regression would show up as small noise on those benches and is easy to miss.
Suggested sequencing
If a single contributor picks these up, lower-numbered issues are higher-value-per-LoC. Suggested order:
This issue is complete when all 6 sub-issues are closed (i.e., each bench has landed). Until then, this issue serves as a single reference point for "what does post-PR-1228 bench coverage need."
Real-world dataset benches (e.g., the dblp data in fluree-db-cli bulk import work). Useful complement to synthetic generators but a separate concern; spin a new issue if/when needed.
iai-callgrind exploration — covered by the bench-infrastructure plan §11 open question; revisit only after we have flap data from criterion-on-ubuntu-latest.
Overview
Parent tracker for the bench-coverage gaps surfaced during review of PR #1228 (
feature/stable-bench— bench foundation). PR #1228 ships the chassis (fluree-bench-support), 7 new hot-path benches, and thebench-gateCI job. The reviewer flagged eight perf-regression-prone hot paths that PR #1228 doesn't cover — bundled here into 6 sub-issues based on which gaps are best landed together as a single PR.These are all real coverage gaps (each has a memory ID documenting a recent or known perf-sensitive area). The gate the foundation PR lands today is "smoke + reconcile only" — it catches benches that no longer compile or panic, but does not yet do regression comparison. That's
bench-nightly's job once we have committed baselines. The coverage gaps below are independent ofbench-nightly: each is "we should have a bench at all for this hot path."Sub-issues
query_hot_bsbmwouldn't catch a regression in theanalyze_property_join_planweighted-width gate unless the wins disappeared end-to-end visibly. Highest-priority of the bunch.lex_sorted_string_idsflag preservation through bulk import + first post-import write). Silent regression vector; would slow production ORDER BY queries on freshly imported datasets.query_hot_bsbmexercises only macroscopically; targeted benches lock in the planner's gate decisions.PreparedBoolExpressioncache regression bench. Per-row cacheability analysis is the easy thing to accidentally re-run; the regression vector is silent and filter-heavy queries are common in production.DataSetDbhistory query coverage. The Query API has a rich layering (db()/db_at_t()/db_at(),DataSetDb) — none of PR Workspace-Wide Benchmarking Foundation #1228's benches exercise it.import_bulkandtransact_commit; a parser regression would show up as small noise on those benches and is easy to miss.Suggested sequencing
If a single contributor picks these up, lower-numbered issues are higher-value-per-LoC. Suggested order:
Definition of done for this parent
This issue is complete when all 6 sub-issues are closed (i.e., each bench has landed). Until then, this issue serves as a single reference point for "what does post-PR-1228 bench coverage need."
References
feature/stable-bench, bench foundation): the chassis + 7 new benches + bench-gate CI job.pr-1228-review.md(in-repo or comment thread). The "Suggested coverage gaps" section enumerates these 8 items.BENCHMARKING.md,docs/contributing/benches.md,fluree-bench-support/README.md— contributor onboarding for adding a bench..claude/proposed-work/docs/plan-benchmark-infrastructure.md— design rationale for the chassis these benches plug into.What this parent issue does NOT track
bench-nightly(separate PR / branch).fluree-db-clibulk import work). Useful complement to synthetic generators but a separate concern; spin a new issue if/when needed.ubuntu-latest.