Skip to content

[bench] Post-PR-#1228 coverage gaps (parent tracker) #1235

@aaj3f

Description

@aaj3f

Overview

Parent tracker for the bench-coverage gaps surfaced during review of PR #1228 (feature/stable-bench — bench foundation). PR #1228 ships the chassis (fluree-bench-support), 7 new hot-path benches, and the bench-gate CI job. The reviewer flagged eight perf-regression-prone hot paths that PR #1228 doesn't cover — bundled here into 6 sub-issues based on which gaps are best landed together as a single PR.

These are all real coverage gaps (each has a memory ID documenting a recent or known perf-sensitive area). The gate the foundation PR lands today is "smoke + reconcile only" — it catches benches that no longer compile or panic, but does not yet do regression comparison. That's bench-nightly's job once we have committed baselines. The coverage gaps below are independent of bench-nightly: each is "we should have a bench at all for this hot path."

Sub-issues

Suggested sequencing

If a single contributor picks these up, lower-numbered issues are higher-value-per-LoC. Suggested order:

  1. [bench] Property-join planner regression bench (analyze_property_join_plan boundaries) #1229 (property-join planner). Largest recent win, biggest blast radius.
  2. [bench] Lex-sorted-string ORDER BY fast-path regression bench #1230 (lex-sorted ORDER BY). Smallest bench (~100–150 LoC), depends on a single invariant; quick to land.
  3. [bench] Scan fast-path regression benches: batched object-join + group-count-firsts #1231 (scan fast-paths). Bundles two related concerns; biggest issue but also biggest macro-coverage gain.
  4. [bench] Filter PreparedBoolExpression cache regression bench #1232 (filter prepared-cache). Filter-heavy queries are a known production pattern.
  5. [bench] Time-travel and multi-ledger DataSetDb history query coverage #1233 (time-travel + multi-ledger). API-surface coverage; lower priority than the targeted regression benches above but higher than parser micros because the API surface is user-visible.
  6. [bench] Parser micro-benches (Turtle, JSON-LD, SPARQL) #1234 (parser micros). Lowest urgency since the parser hot paths are exercised macro-visibly.

Definition of done for this parent

This issue is complete when all 6 sub-issues are closed (i.e., each bench has landed). Until then, this issue serves as a single reference point for "what does post-PR-1228 bench coverage need."

References

  • PR Workspace-Wide Benchmarking Foundation #1228 (feature/stable-bench, bench foundation): the chassis + 7 new benches + bench-gate CI job.
  • PR review: pr-1228-review.md (in-repo or comment thread). The "Suggested coverage gaps" section enumerates these 8 items.
  • BENCHMARKING.md, docs/contributing/benches.md, fluree-bench-support/README.md — contributor onboarding for adding a bench.
  • Bench-infrastructure plan: .claude/proposed-work/docs/plan-benchmark-infrastructure.md — design rationale for the chassis these benches plug into.

What this parent issue does NOT track

  • Regression comparison enforcement — that's bench-nightly (separate PR / branch).
  • Real-world dataset benches (e.g., the dblp data in fluree-db-cli bulk import work). Useful complement to synthetic generators but a separate concern; spin a new issue if/when needed.
  • iai-callgrind exploration — covered by the bench-infrastructure plan §11 open question; revisit only after we have flap data from criterion-on-ubuntu-latest.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions