nbrs

High-performance workload generation and database testing in Rust.

nbrs generates deterministic, reproducible request streams at scale. Every value is derived from a cycle number through a composable DAG of functions — same cycle, same output, every time. This makes workloads debuggable, cacheable, and exactly reproducible across runs.

Part of the nosqlbench project.

This system was derived from things learned building nosqlbench, and shares many of its concepts. However, some things were kept and some removed. This will be a much leaner and meaner version of what nosqlbench is. It will do some things differently.

Quick Start

$ nbrs run op='INSERT INTO t (id, name) VALUES ({{mod(hash(cycle), 1000000)}}, "{{number_to_words(cycle)}}")' cycles=5

INSERT INTO t (id, name) VALUES (527897, "zero")
INSERT INTO t (id, name) VALUES (460078, "one")
INSERT INTO t (id, name) VALUES (564547, "two")
INSERT INTO t (id, name) VALUES (960189, "three")
INSERT INTO t (id, name) VALUES (862456, "four")

Or from a workload file:

#!/usr/bin/env nbrs
# service.yaml

params:
  keyspace: demo
  table: users
  user_count: "100000"

bindings: |
  input cycle: u64
  user_id := mod_wire(hash(cycle), user_count)
  user_name := number_to_words(mod(hash(hash(cycle)), 1000))
  is_write := mod(cycle, 5)

ops:
  read_user:
    ratio: 4
    stmt: "SELECT * FROM {keyspace}.{table} WHERE id={user_id}"
  write_user:
    ratio: 1
    if: is_write
    stmt: "INSERT INTO {keyspace}.{table} (id, name) VALUES ({user_id}, '{user_name}')"

$ chmod +x service.yaml
$ ./service.yaml cycles=100 concurrency=4 rate=1000

Features

Generation Kernel (Polydat) — A DAG-based data generation engine with:

• Infix operators (+, -, *, /, %, **, &, |, ^, <<, >>)
• 100+ node functions: hash, distributions, noise, strings, vectors, CSV/JSONL
• Type-aware dispatch with auto-widening (u64/f64/string)
• Constant folding, provenance-based invalidation, JIT compilation
• Module system with composable .polydat files and stdlib

Workload Engine — Flexible execution with:

• Phased workloads (schema → rampup → steady-state)
• Conditional ops (if: field skips per-cycle)
• Latency injection (delay: field for GK-driven think time)
• Ratio-weighted op sequencing
• Capture flow between ops within a stanza
• Polydat expressions in config (cycles="{vector_count("example")}")

Adapters — Protocol drivers for:

• stdout (debugging, dry-run, format=json/csv/stmt)
• HTTP (REST APIs, configurable timeouts)
• CQL (Cassandra/ScyllaDB — cassandra-cpp or scylla driver)
• testkit (simulated service: latency, errors, capacity limits)

Observability — Built-in metrics and dashboards:

• HDR histograms for latency percentiles
• OpenMetrics push to Prometheus/VictoriaMetrics
• Live TUI dashboard (--tui)
• Web dashboard (nbrs web)

Build

cargo build --release

Enable shell completions:

eval "$(nbrs completions)"

Commands

nbrs run workload=file.yaml cycles=1M concurrency=8 rate=10000
nbrs run op='hello {{hash(cycle)}}' cycles=10
nbrs bench wiring 'hash(cycle)' cycles=1M threads=1:8*2
nbrs describe wiring functions
nbrs report plot workload=file.yaml   # render the workload's report: items
nbrs metrics list                     # introspect a session's metrics db
nbrs web --daemon

Data Wiring

The wiring for nbrs is provided by a dedicated subsystem called polydat. This takes the place of the classic virtual dataset procedural generation system in nosqlbench, but it is, like nbrs, derived and evolved from the original blueprints around lessons learned in that project.

Visualizing wiring expressions

nbrs wiring visualize <expr> evaluates a wiring expression across a range of cycles and plots it in the terminal. The output wire names choose the plot mode:

# default — each output plotted against cycle (line plot)
nbrs wiring visualize 'y := sin(to_f64(cycle) * 0.1)' cycles=200

# wires named x / y → parametric (this traces a circle)
nbrs wiring visualize 't := to_f64(cycle)*0.06; x := cos(t); y := sin(t)' cycles=120

# wires named r / theta → polar (this traces a 3-petal rose)
nbrs wiring visualize 'theta := to_f64(cycle)*0.06; r := cos(theta*3.0)' cycles=120

r/theta also accept radius/angle or rho/phi. Pass --mode=plot|parametric|polar to override the name inference.

wiring visualize is sugar for nbrs run adapter=plotter render=single — it builds a one-op plotter workload and runs it through the engine. The plotter adapter renders either a single static snapshot (the default for visualize, and for any non-TTY output) or live, animated as cycles arrive — nbrs run adapter=plotter op='…' render=live (or render=5hz for a specific refresh rate; a TTY defaults to live).

Examples

See examples/:

• workloads/ — runnable workload YAMLs: phases, conditional ops (if:), delays, scenarios, cursors & partitions, capture flow, comprehensions, dynamic controls, and reports
• modules/ — the Polydat module system (.polydat files imported by module workloads)

Architecture

polydat     Polydat engine: DAG compilation, node functions, JIT, provenance
nbrs-workload     YAML parsing, bind points, inline expressions, phasing
nbrs-runtime     Async execution engine, dispenser wrappers, capture flow
nbrs-metrics      HDR histograms, frame capture, OpenMetrics export
nbrs-rate         Async token bucket rate limiter
nbrs-errorhandler Composable error routing
nb-rs           CLI binary (nbrs), bench, plot, web dashboard
nbrs-tui          Terminal UI for live monitoring
nbrs-web          Web dashboard with Axum + HTMX

Functional Areas

Each of these areas has a distinctive design which has evolved from its nosqlbench form. This represents a capsule-form view of the user-facing elements which compose together to make a whole system. Each of these should be considered a separate, modular subsystem.

They are due for some documentation.

• user interface (readout, output channels, TUI)
  • TUI status views (readout)
  • CLI dynamic completion help
• test data / variates (polydat, data access libraries)
• custom op behaviors (op wrapping facility and yaml alignment)
• workload description (session/workload/scenario/phase)
• dynamic controls (realtime hooks, api, usage examples)
• reports / plots / tables (per-session, per-workload, ...)
• metrics db / metricsql (stores, query interface, uniformity)
• portability / local-only sufficiency
• safety checks

License

Apache-2.0