CertBench

CertBench is an open-source toolkit for auditing protocol-induced spatial leakage in patch-based visual recognition. It implements the protocol-coupling certification pipeline used in the accompanying manuscript:

Gamma(P) -> L_alpha(P) -> C_alpha(D) -> I_m

The current implementation supports hyperspectral image classification benchmarks and provides reusable primitives for remote-sensing tiles and medical whole-slide patches.

What Is Included

• certbench/: protocol-coupling certification API.
• leak_bound/: legacy-compatible leakage-bound implementation used by the paper scripts.
• scripts/: experiment, table-generation, and audit scripts.
• tests/: smoke tests for core leakage calculations.
• results/: lightweight JSON/TEX result artifacts used by the manuscript.
• paper/: LaTeX sources and generated tables.

Large benchmark data files are not included. Use scripts/download_datasets.sh or provide local .npy/.mat files.

Installation

git clone https://github.com/Marchematics/CertBench.git
cd CertBench
pip install -e .

For a Conda environment:

conda env create -f environment.yml
conda activate certbench
pip install -e .

Command Line Usage

certbench audit \
  --image data/indian_pines.npy \
  --labels data/indian_pines_labels.npy \
  --name "Indian Pines" \
  --n-train 1000 \
  --patch-radius 6 \
  --output indian_pines_audit.json

The tool reports the protocol coupling, hard-overlap probability, estimated correlation length, certified buffer scale, certifiability value, and certification regime.

Python API

from certbench.coupling import protocol_coupling
from certbench.certifiability import certifiability_functional

gamma = protocol_coupling(
    protocol="RS",
    H=145,
    W=145,
    patch_radius=6,
    n_train=1000,
    ell=6.5,
)

cert = certifiability_functional(
    H=145,
    W=145,
    labels=labels,
    ell=6.5,
    patch_radius=6,
    n_train=1000,
    epsilon=0.05,
)

Reproducing Paper Tables

python scripts/make_extended_tables.py
python scripts/generate_sensitivity_tables.py

The historical census and Experiment 5 tables are included as lightweight artifacts under results/ and paper/.

Data Policy

This repository intentionally excludes raw hyperspectral benchmark files (.mat, .npy, .hdr, .tif, etc.). The scripts expect users to download datasets from their official sources and convert them locally.

License

MIT License.