Date: Wed, 8 Jul 2026 16:34:25 -0400
Subject: [PATCH 1/7] Fix documentation and README issue
---
.gitignore | 3 +++
README.md | 2 +-
documentation/documentation.md | 1 +
3 files changed, 5 insertions(+), 1 deletion(-)
diff --git a/.gitignore b/.gitignore
index 30aa957..765b129 100644
--- a/.gitignore
+++ b/.gitignore
@@ -27,6 +27,9 @@ documentation/readthedocs/_build/
# PACKAGE DATA
# -------------------------------
+# Dev TODOs
+future_todos.md
+
# Ignore everything inside OUTPUT, but keep the folder itself
src/shinier/data/OUTPUT/*
!src/shinier/data/OUTPUT/.gitkeep
diff --git a/README.md b/README.md
index 50bce5c..e57339e 100644
--- a/README.md
+++ b/README.md
@@ -102,7 +102,7 @@ Change the mode number (e.g. `opt = Options(mode=3)`) to change image processing
| 6 | `hist_match → spec_match` | Histogram, then spectrum |
| 7 | `sf_match → hist_match` | Spatial frequency, then histogram |
| 8 | `spec_match → hist_match` (default) | Spectrum, then histogram (recommended) |
-| 9 | `dithering` | Dithering only |
+| 9 | `ie_methods` or `dithering` | Standalone per-image transform (histogram-derived enhancement or dithering) |
Below is an example of results obtained using mode 5 with joint histogram equalization and spatial frequency normalization.
diff --git a/documentation/documentation.md b/documentation/documentation.md
index 60ad522..a711945 100644
--- a/documentation/documentation.md
+++ b/documentation/documentation.md
@@ -367,6 +367,7 @@ Applies a standalone transform to each image independently — no inter-image ta
Histogram equalization can be achieved through **Exact Histogram Specification (EHS)** using a flat, uniform target histogram (`target_hist="equal"`, `mode=2` or modes 5–8). Pixels are individually ranked and assigned to target bins, allowing the output to exactly match the feasible discrete uniform histogram.
SHINIER also provides **histogram-derived methods** (`mode=9`, `standalone_op="ie_methods"`), including `classic_he`, `tidhe`, and `rdfhe`. These methods compute gray-level mappings from the image histogram or CDF. Because identical input intensities receive the same output value, the resulting histogram is generally only approximately uniform.
+
---
### Border Artifacts and FFT Padding
From 4dc7e34ded9db08528e1a9288740757aed388a7c Mon Sep 17 00:00:00 2001
From: Kaapra
Date: Thu, 9 Jul 2026 14:16:02 -0400
Subject: [PATCH 2/7] Reference for bp2bpsim
---
src/shinier/utils.py | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/src/shinier/utils.py b/src/shinier/utils.py
index da9d94d..92918e9 100644
--- a/src/shinier/utils.py
+++ b/src/shinier/utils.py
@@ -5013,6 +5013,12 @@ def compute_bp2bpsim(reference: np.ndarray, enhanced: np.ndarray, n_bits: int =
Values range from 0 to 1. Higher values indicate more matching bits across
corresponding pixels and channels.
+
+ References
+ ----------
+ Rahman, H., & Paul, G. C. (2023). Tripartite sub-image histogram equalization for slightly
+ low contrast gray-tone image enhancement. Pattern Recognition, 134, Article 109043.
+ https://doi.org/10.1016/j.patcog.2022.109043
"""
_check_same_shape(reference, enhanced, "compute_bp2bpsim")
if n_bits < 1 or n_bits > 8:
From 0b357ccb0c68fd0a6d4a642bca6e36851a92dd18 Mon Sep 17 00:00:00 2001
From: Kaapra
Date: Mon, 13 Jul 2026 16:14:27 -0400
Subject: [PATCH 3/7] Add script for comparing SHINIER (Python) with MATLAB
SHINE toolbox
- Introduced `run_matlab_shine_comparison.sh` to facilitate comparison between SHINIER and the original MATLAB SHINE toolbox.
- The script includes checks for required executables and directories, and prompts the user for necessary paths.
- Implements validation for the SHINE toolbox and its functions.
- Supports configurable parameters such as modes, iterations, and output directories.
- Automates the process of running MATLAB scripts and comparing outputs with Python.
---
.gitignore | 1 +
documentation/documentation.md | 12 +
src/shinier/ImageProcessor.py | 2 +-
src/shinier/utils.py | 2 +-
tests/README.md | 44 +
tests/tools/matlab_shine_comparison.py | 1293 ++++++++++++++++++++
tests/tools/run_matlab_shine_comparison.sh | 329 +++++
7 files changed, 1681 insertions(+), 2 deletions(-)
create mode 100644 tests/tools/matlab_shine_comparison.py
create mode 100755 tests/tools/run_matlab_shine_comparison.sh
diff --git a/.gitignore b/.gitignore
index 765b129..a3f52eb 100644
--- a/.gitignore
+++ b/.gitignore
@@ -14,6 +14,7 @@ __pycache__/
*.pyd
*.dll
.pytest_tmp/
+tmp/
*venv*/
*.db
*.db*
diff --git a/documentation/documentation.md b/documentation/documentation.md
index a711945..7c1f46f 100644
--- a/documentation/documentation.md
+++ b/documentation/documentation.md
@@ -857,6 +857,18 @@ See `tests/README.md` for a detailed breakdown.
`ImageEnhancement_validation_test.py` validates each image-enhancement algorithm (TIDHE, RDFHE, NFLDICE, BETCE, SFCEF) against pixel-exact MATLAB reference outputs stored as SHA-256 hashes in `tests/assets/image_enhancement_matlab_sha256.json`.
SFCEF uses a pixel-difference bound (`max_diff ≤ 1`) instead of exact hash equality due to FMA-induced rounding differences between MATLAB and NumPy.
+### MATLAB SHINE Comparison
+
+A standalone tool benchmarks SHINIER directly against the original [MATLAB SHINE toolbox](http://www.mapageweb.umontreal.ca/gosselif/SHINE/) across processing modes 1–8.
+It compares three implementations — `matlab_shine` (the original toolbox), `shinier_legacy` (`legacy_mode=True`, MATLAB-compatible behavior), and `shinier_modern_gray` (SHINIER defaults on grayscale) — in two stages: pixel differences between saved outputs, and distances to shared fixed targets (histogram and spectrum), each measured in the implementation's own processing domain.
+
+```bash
+# Requires MATLAB and the SHINE toolbox
+bash tests/tools/run_matlab_shine_comparison.sh
+```
+
+Results are written as CSV files under `tmp/matlab_shine_comparison/` and summarized in terminal tables. See `tests/README.md` and the docstring of `tests/tools/matlab_shine_comparison.py` for details.
+
---
## 📚 Usage Examples
diff --git a/src/shinier/ImageProcessor.py b/src/shinier/ImageProcessor.py
index 5a15e63..da83e92 100644
--- a/src/shinier/ImageProcessor.py
+++ b/src/shinier/ImageProcessor.py
@@ -409,7 +409,7 @@ def _compute_initial_target_histogram(self, n_bins: int = 256):
target_hist = self._compute_target_hist_from_image_path(target_hist, n_bins=n_bins)
else:
target_hist = target_hist[:, None] if target_hist.ndim == 1 else target_hist
- target_hist /= (target_hist.sum(axis=0, keepdims=True) + 1e-12)
+ target_hist = target_hist / (target_hist.sum(axis=0, keepdims=True) + 1e-12)
if target_hist.shape[0] != n_bins:
raise ValueError(f"target_hist must have {n_bins} bins, but has {target_hist.shape[0]}.")
if target_hist.ndim > 1 and target_hist.shape[-1] != self.dataset.buffer.n_channels:
diff --git a/src/shinier/utils.py b/src/shinier/utils.py
index 92918e9..c33cadc 100644
--- a/src/shinier/utils.py
+++ b/src/shinier/utils.py
@@ -1023,7 +1023,7 @@ def get_radius_grid(x_size: int, y_size: int, legacy_mode: bool = False) -> np.n
# --- polar radius, MATLAB rounding rule ---
r = np.hypot(XX, YY)
r_adjustment = -1 if (x_size % 2 == 1) or (y_size % 2 == 1) else 0
- r = MatlabOperators.round(r) if legacy_mode else np.round(r, decimals=0) + r_adjustment
+ r = (MatlabOperators.round(r) if legacy_mode else np.round(r, decimals=0)) + r_adjustment
# Non-negative integer bin indices
return np.clip(r, 0, None).astype(np.int64)
diff --git a/tests/README.md b/tests/README.md
index 48ff02b..3f8abd1 100644
--- a/tests/README.md
+++ b/tests/README.md
@@ -314,3 +314,47 @@ This will rebuild the same `Options`, reload selected images, and re-run the fai
```bash
python -m tests.tools.replay_failure path/to/failure_xxxxx.pkl
```
+
+---
+
+## 🔬 MATLAB SHINE Comparison
+
+SHINIER ships with a standalone comparison tool that benchmarks the Python
+implementations against the original
+[MATLAB SHINE toolbox](http://www.mapageweb.umontreal.ca/gosselif/SHINE/)
+across processing modes 1–8. Three implementations are compared:
+
+| Implementation | Description |
+|-----------------------|--------------------------------------------------------------------|
+| `matlab_shine` | Original MATLAB SHINE toolbox, driven by a generated MATLAB script |
+| `shinier_legacy` | SHINIER with `legacy_mode=True` (MATLAB-compatible behavior) |
+| `shinier_modern_gray` | SHINIER defaults on grayscale (xyY luminance processing) |
+
+The tool produces two comparison stages:
+
+1. **Output comparison** — pixel differences (RMSE, MAE, max abs, equal
+ fraction, histogram L1) between saved MATLAB and Python images.
+2. **Fixed-target comparison** — every implementation receives the same fixed
+ initial Python targets (histogram and spectrum) and each output is measured
+ against the target in its own processing domain.
+
+Requirements: a local MATLAB installation and the SHINE toolbox.
+
+```bash
+# Complete run (asks for MATLAB/SHINE paths if not found)
+bash tests/tools/run_matlab_shine_comparison.sh
+
+# Common overrides
+MATLAB_BIN=/Applications/MATLAB_R2025a.app/bin/matlab \
+SHINE_DIR=~/toolboxes/shinetoolbox \
+MODES="2 3 4" LIMIT=8 ITERATIONS=5 \
+bash tests/tools/run_matlab_shine_comparison.sh
+
+# Keep all intermediate images, MATLAB scripts and .mat files
+FULL_TRACKING=1 bash tests/tools/run_matlab_shine_comparison.sh
+```
+
+Results are written as CSV files (summary and per-image detail) under
+`tmp/matlab_shine_comparison/`, and summary tables are printed to the
+terminal. See the module docstring of
+`tests/tools/matlab_shine_comparison.py` for the full metric definitions.
diff --git a/tests/tools/matlab_shine_comparison.py b/tests/tools/matlab_shine_comparison.py
new file mode 100644
index 0000000..a1624d5
--- /dev/null
+++ b/tests/tools/matlab_shine_comparison.py
@@ -0,0 +1,1293 @@
+"""Compare SHINIER (Python) against the original MATLAB SHINE toolbox.
+
+Run the complete comparison with:
+
+ bash tests/tools/run_matlab_shine_comparison.sh
+
+Requirements: MATLAB and the SHINE toolbox
+(http://www.mapageweb.umontreal.ca/gosselif/SHINE/).
+
+Three implementations are compared across processing modes 1-8:
+
+- ``matlab_shine``: the original MATLAB SHINE toolbox (lumMatch, histMatch,
+ sfMatch, specMatch), driven by a generated MATLAB script that replicates
+ the ``SHINE.m`` composite-mode loop;
+- ``shinier_legacy``: SHINIER with ``legacy_mode=True`` (MATLAB-compatible
+ grayscale conversion, rounding, and noise tie-breaking);
+- ``shinier_modern_gray``: SHINIER defaults on grayscale (xyY luminance
+ processing, hybrid tie-breaking, sRGB-encoded export).
+
+Two comparison stages are produced:
+
+1. Output comparison: pixel differences between saved MATLAB and Python
+ images (RMSE, MAE, max abs, equal fraction, histogram L1).
+2. Fixed-target comparison: every implementation receives the same fixed
+ initial Python targets (histogram and spectrum), and each output is
+ measured against the target in its own processing domain. MATLAB and
+ ``shinier_legacy`` share the legacy target; ``shinier_modern_gray`` uses
+ its own target.
+
+Default output is CSV-only; use ``FULL_TRACKING=1`` or ``--full-tracking`` to
+keep generated inputs, PNGs, MATLAB scripts, and ``.mat`` files. The grayscale
+CSV compares MATLAB's input gray image to SHINIER's internal
+``ImageProcessor._initial_buffer``. PNG target metrics map saved images back
+through each implementation's grayscale domain; ``png_hist_l1`` also uses
+``rounded_target_hist`` and the export/import bin transform. SHINIER rows
+include ``soft_clip``; MATLAB rows use SHINE's rescale/uint8 behavior.
+Terminal tables use scientific notation with 3 decimals; CSVs keep full
+precision.
+"""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import os
+import shutil
+import subprocess
+import sys
+import tempfile
+import textwrap
+from datetime import datetime
+from pathlib import Path
+
+import numpy as np
+from PIL import Image
+from scipy.io import savemat
+
+REPO_ROOT = Path(__file__).resolve().parents[2]
+DEFAULT_SHINE_DIR = Path(os.environ["SHINE_DIR"]) if os.environ.get("SHINE_DIR") else None
+DEFAULT_MATLAB = Path(os.environ.get("MATLAB_BIN", "matlab"))
+DEFAULT_OUTPUT_DIR = REPO_ROOT / "tmp/matlab_shine_comparison"
+
+if "MPLCONFIGDIR" not in os.environ:
+ mpl_cache = Path(tempfile.gettempdir()) / "shinier_matplotlib"
+ mpl_cache.mkdir(parents=True, exist_ok=True)
+ os.environ["MPLCONFIGDIR"] = str(mpl_cache)
+
+sys.path.insert(0, str(REPO_ROOT / "src"))
+
+from shinier import ImageDataset, ImageProcessor, Options # noqa: E402
+from shinier.color.Converter import ColorConverter, rgb2gray # noqa: E402
+from shinier.utils import ( # noqa: E402
+ _crop_after_fft,
+ compute_rmse,
+ compute_tvd_hist,
+ get_radius_grid,
+ image_spectrum,
+ MatlabOperators,
+ pol2cart,
+ rounded_target_hist,
+ rotational_avg,
+ soft_clip,
+ uint8_plus,
+)
+
+TARGET_REFERENCE = (
+ "Initial Python target. MATLAB and SHINIER legacy use the legacy target; "
+ "SHINIER modern-gray uses its own modern-gray target. Composite modes are "
+ "evaluated against this fixed initial target. In this target-injection run, "
+ "MATLAB and Python are given the fixed target; normal composite processing "
+ "without injected targets may use moving targets internally."
+)
+ANSI_BOLD_RED = "\033[1;31m"
+ANSI_RESET = "\033[0m"
+MATLAB_IMPLEMENTATION = "matlab_shine"
+LEGACY_IMPLEMENTATION = "shinier_legacy"
+MODERN_GRAY_IMPLEMENTATION = "shinier_modern_gray"
+PYTHON_IMPLEMENTATIONS = {
+ LEGACY_IMPLEMENTATION: ("legacy", True),
+ MODERN_GRAY_IMPLEMENTATION: ("modern_gray", False),
+}
+
+HIST_MODES = {2, 5, 6, 7, 8}
+SF_MODES = {3, 5, 7}
+SPECTRUM_MODES = {4, 6, 8}
+TARGET_MODES = HIST_MODES | SF_MODES | SPECTRUM_MODES
+
+TARGET_METRIC_LABELS = {
+ "internal_buffer_hist_l1_to_python_target": "int_hist_l1",
+ "exported_png_hist_l1_to_python_target": "png_hist_l1",
+ "internal_buffer_sf_rmse_to_python_target": "int_sf_rmse",
+ "exported_png_sf_rmse_to_python_target": "png_sf_rmse",
+ "pre_range_fourier_spectrum_rmse_to_python_target": "fft_spec",
+ "pre_range_image_spectrum_rmse_to_python_target": "pre_spec",
+ "pre_range_out_of_range_fraction": "pre_oor",
+ "post_soft_clip_spectrum_rmse_to_python_target": "clip_spec",
+ "internal_buffer_spectrum_rmse_to_python_target": "int_spec_rmse",
+ "exported_png_spectrum_rmse_to_python_target": "png_spec_rmse",
+}
+PIXEL_COLUMNS = "images mean_rmse mean_mae max_abs mean_equal_fraction mean_hist_l1".split()
+OUTPUT_COMPARISON_COLUMNS = "mode reference python_output".split() + PIXEL_COLUMNS[1:]
+TARGET_TABLE_COLUMNS = ["mode", "implementation", "target"] + [
+ f"mean_{key}" for key in TARGET_METRIC_LABELS
+]
+TARGET_TABLE_LABELS = {f"mean_{key}": label for key, label in TARGET_METRIC_LABELS.items()}
+# CSV-only keys extend the displayed metrics with the signed out-of-range fractions.
+TARGET_METRIC_KEYS = (
+ *TARGET_METRIC_LABELS,
+ "pre_range_below_zero_fraction",
+ "pre_range_above_one_fraction",
+)
+TARGET_TABLE_NOTES = [
+ "Legend:",
+ " int_* = metric on SHINIER ImageProcessor._final_buffer; blank for MATLAB.",
+ " png_* = exported PNG mapped back to the implementation target domain.",
+ " fft/pre/clip_spec = SHINIER-only spectrum checks before/after soft_clip; pre_oor = pre-correction out-of-range fraction.",
+ " clip_spec vs int_spec_rmse gap = final rescaling for modes 4/6, final hist_match for mode 8.",
+ " SHINIER uses soft_clip, which lowers hard clipping but can worsen fixed-target RMSE; MATLAB uses SHINE rescale/uint8.",
+ " png_hist_l1 uses rounded_target_hist plus the same export/import bin transform.",
+]
+TARGET_WARNING_LINES = [
+ "target metrics use fixed initial Python targets; normal composite runs may use moving targets internally.",
+ "MATLAB/legacy share the legacy target; modern-gray uses its own target.",
+ "SHINIER rows include soft_clip; MATLAB rows use SHINE rescale/uint8.",
+ "png_hist_l1 uses rounded_target_hist plus the export/import bin mapping.",
+]
+RUN_INFO_KEYS = [
+ "run_root",
+ "matlab_runner",
+ "modes",
+ "images",
+ "use_python_targets",
+ "skip_matlab",
+ "prepare_only",
+]
+INPUT_GRAY_TITLE = "MATLAB input grayscale vs SHINIER internal initial buffer"
+TargetSet = dict[str, np.ndarray | int | str | bool]
+TargetSets = dict[str, TargetSet]
+
+
+def parse_args() -> argparse.Namespace:
+ parser = argparse.ArgumentParser(
+ description=__doc__,
+ formatter_class=argparse.RawDescriptionHelpFormatter,
+ )
+ add = parser.add_argument
+ add("--input-dir", type=Path, default=REPO_ROOT / "tests/assets/SAMPLE_64X64")
+ add("--output-dir", type=Path, default=DEFAULT_OUTPUT_DIR)
+ add("--run-root", type=Path)
+ add("--shine-dir", type=Path, default=DEFAULT_SHINE_DIR)
+ add("--matlab-bin", type=Path, default=DEFAULT_MATLAB)
+ add("--modes", type=int, nargs="+", default=list(range(1, 9)))
+ add("--iterations", type=int, default=5)
+ add("--seed", type=int, default=42)
+ add("--limit", type=int, default=8)
+ for flag in (
+ "--skip-matlab",
+ "--prepare-only",
+ "--use-python-targets",
+ "--quiet-run-info",
+ "--full-tracking",
+ ):
+ add(flag, action="store_true")
+ return parser.parse_args()
+
+
+def quote_matlab_path(path: Path) -> str:
+ return str(path).replace("'", "''")
+
+
+def prepare_inputs(source_dir: Path, output_root: Path, limit: int) -> list[Path]:
+ input_dir = output_root / "input"
+ input_dir.mkdir(parents=True, exist_ok=True)
+ paths = sorted(source_dir.glob("*.png"))[:limit]
+ if not paths:
+ raise FileNotFoundError(f"No PNG inputs found in {source_dir}")
+ for path in paths:
+ shutil.copy2(path, input_dir / path.name)
+ return [input_dir / path.name for path in paths]
+
+
+def write_matlab_runner(
+ *,
+ output_root: Path,
+ input_dir: Path,
+ shine_dir: Path | None,
+ modes: list[int],
+ iterations: int,
+ seed: int,
+ use_python_targets: bool = False,
+) -> Path:
+ runner = output_root / "run_matlab_shine.m"
+ modes_text = " ".join(str(mode) for mode in modes)
+ shine_addpath = f"addpath('{quote_matlab_path(shine_dir)}');" if shine_dir else ""
+ shine_check = (
+ "required_shine = {'lumMatch', 'histMatch', 'sfMatch', 'specMatch'};\n"
+ "missing_shine = {};\n"
+ "for si = 1:numel(required_shine)\n"
+ " if exist(required_shine{si}, 'file') ~= 2\n"
+ " missing_shine{end + 1} = required_shine{si}; %#ok\n"
+ " end\n"
+ "end\n"
+ "if ~isempty(missing_shine)\n"
+ " error(['Missing SHINE functions: ', strjoin(missing_shine, ', '), ...\n"
+ " '. Set SHINE_DIR to the SHINE toolbox folder or download SHINE from ', ...\n"
+ " 'http://www.mapageweb.umontreal.ca/gosselif/SHINE/']);\n"
+ "end"
+ )
+ target_load = ""
+ hist_arg = ""
+ spectrum_arg = ""
+ if use_python_targets:
+ target_load = (
+ f"targets = load('{quote_matlab_path(output_root / 'python_targets.mat')}');"
+ )
+ hist_arg = ", targets.target_hist_counts"
+ spectrum_arg = ", targets.target_spectrum"
+ runner.write_text(
+ textwrap.dedent(
+ f"""
+ {shine_addpath}
+ {shine_check}
+ input_dir = '{quote_matlab_path(input_dir)}';
+ output_root = '{quote_matlab_path(output_root / "matlab")}';
+ modes = [{modes_text}];
+ {target_load}
+ files = dir(fullfile(input_dir, '*.png'));
+ [~, order] = sort({{files.name}});
+ files = files(order);
+
+ original = cell(1, numel(files));
+ input_gray_dir = fullfile(output_root, 'input_gray');
+ if ~exist(input_gray_dir, 'dir')
+ mkdir(input_gray_dir);
+ end
+ for k = 1:numel(files)
+ im = imread(fullfile(input_dir, files(k).name));
+ if ndims(im) == 3
+ im = rgb2gray(im);
+ end
+ original{{k}} = im;
+ imwrite(uint8(im), fullfile(input_gray_dir, files(k).name));
+ end
+
+ hist_corr_summary = {{}};
+ for mi = 1:numel(modes)
+ mode = modes(mi);
+ rand('seed', {seed});
+ images = original;
+ if mode >= 5
+ n_iter = {iterations};
+ else
+ n_iter = 1;
+ end
+ for iter = 1:n_iter
+ switch mode
+ case 1
+ images = lumMatch(images);
+ case {{2, 5, 6}}
+ [shine_log, images] = evalc('histMatch(images, 0{hist_arg})');
+ hist_corr_summary = append_hist_corr( ...
+ hist_corr_summary, mode, iter, 'hist_before_spectrum', shine_log);
+ end
+ switch mode
+ case {{3, 5, 7}}
+ images = sfMatch(images, 1{spectrum_arg});
+ case {{4, 6, 8}}
+ images = specMatch(images, 1{spectrum_arg});
+ end
+ switch mode
+ case {{7, 8}}
+ [shine_log, images] = evalc('histMatch(images, 0{hist_arg})');
+ hist_corr_summary = append_hist_corr( ...
+ hist_corr_summary, mode, iter, 'hist_after_spectrum', shine_log);
+ end
+ end
+
+ out_dir = fullfile(output_root, sprintf('mode_%d', mode));
+ if ~exist(out_dir, 'dir')
+ mkdir(out_dir);
+ end
+ for k = 1:numel(files)
+ imwrite(uint8(images{{k}}), fullfile(out_dir, files(k).name));
+ end
+ end
+
+ print_hist_corr_summary(hist_corr_summary);
+
+ function rows = append_hist_corr(rows, mode, iter, stage, shine_log)
+ tokens = regexp( ...
+ shine_log, ...
+ 'Correlation between processed image and target histogram:\\s*([0-9.eE+-]+)', ...
+ 'tokens');
+ if isempty(tokens)
+ return;
+ end
+ values = zeros(1, numel(tokens));
+ for ti = 1:numel(tokens)
+ values(ti) = str2double(tokens{{ti}}{{1}});
+ end
+ rows(end + 1, :) = {{ ...
+ mode, iter, stage, numel(values), min(values), mean(values), max(values) ...
+ }};
+ end
+
+ function print_hist_corr_summary(rows)
+ if isempty(rows)
+ return;
+ end
+ fprintf('\\nMATLAB SHINE histMatch correlation summary\\n');
+ fprintf('------+-----------+----------------------+--------+-----------+-----------+-----------\\n');
+ fprintf('mode | iteration | stage | images | min_corr | mean_corr | max_corr\\n');
+ fprintf('------+-----------+----------------------+--------+-----------+-----------+-----------\\n');
+ for ri = 1:size(rows, 1)
+ fprintf( ...
+ '%-5d | %-9d | %-20s | %-6d | %.3e | %.3e | %.3e\\n', ...
+ rows{{ri, 1}}, rows{{ri, 2}}, rows{{ri, 3}}, rows{{ri, 4}}, ...
+ rows{{ri, 5}}, rows{{ri, 6}}, rows{{ri, 7}});
+ end
+ fprintf('------+-----------+----------------------+--------+-----------+-----------+-----------\\n');
+ end
+ """
+ ).strip()
+ + "\n"
+ )
+ return runner
+
+
+def _build_options(
+ *,
+ input_dir: Path,
+ output_dir: Path,
+ mode: int,
+ iterations: int,
+ seed: int,
+ target_hist: np.ndarray | None = None,
+ target_spectrum: np.ndarray | None = None,
+ legacy_mode: bool = True,
+) -> Options:
+ return Options(
+ input_folder=input_dir,
+ output_folder=output_dir,
+ mode=mode,
+ legacy_mode=legacy_mode,
+ as_gray=True,
+ # Keep False here: True uses equal-channel grayscale; False uses the
+ # rec601 color-treatment path (legacy rgb2gray or modern xyY/Y).
+ linear_luminance=False,
+ rec_standard=1,
+ iterations=iterations,
+ seed=seed,
+ target_hist=target_hist,
+ target_spectrum=target_spectrum,
+ verbose=-1,
+ )
+
+
+def compute_python_targets(
+ *,
+ input_dir: Path,
+ output_root: Path,
+ seed: int,
+ legacy_mode: bool = True,
+ target_name: str = "legacy",
+ export_matlab_targets: bool = True,
+) -> TargetSet:
+ target_dir = output_root / f"python_target_source_{target_name}"
+ target_dir.mkdir(parents=True, exist_ok=True)
+ # Only the initial targets are read from this run; they are computed before
+ # the iteration loop, so a single iteration is enough.
+ options = _build_options(
+ input_dir=input_dir,
+ output_dir=target_dir,
+ mode=8,
+ iterations=1,
+ seed=seed,
+ legacy_mode=legacy_mode,
+ )
+ proc = ImageProcessor(dataset=ImageDataset(options=options), options=options)
+
+ target_hist = np.asarray(proc._initial_targets["hist"], dtype=np.float64)
+ target_hist_1d = target_hist[:, 0] if target_hist.ndim == 2 else target_hist
+ first_image = read_saved_gray(sorted(input_dir.glob("*.png"))[0])
+ n_pixels = int(first_image.size)
+ target_hist_freq = rounded_target_hist(target_hist_1d, n_pixels)
+ target_hist_counts = np.rint(target_hist_freq * n_pixels).astype(np.int64)
+
+ target_spectrum = np.asarray(proc._initial_targets["spectrum"], dtype=np.float64).squeeze()
+ radius = get_radius_grid(
+ target_spectrum.shape[0],
+ target_spectrum.shape[1],
+ legacy_mode=legacy_mode,
+ )
+ target_sf = rotational_avg(target_spectrum, radius)
+
+ if export_matlab_targets:
+ target_path = output_root / "python_targets.mat"
+ savemat(
+ target_path,
+ {
+ "target_hist_freq": target_hist_freq[:, None],
+ "target_hist_counts": target_hist_counts[:, None],
+ "target_spectrum": target_spectrum,
+ "target_sf": target_sf[:, None],
+ "n_pixels": np.array([[n_pixels]], dtype=np.int64),
+ },
+ )
+ (output_root / "python_target_reference.txt").write_text(TARGET_REFERENCE + "\n")
+ return {
+ "target_name": target_name,
+ "legacy_mode": legacy_mode,
+ "target_hist_freq": target_hist_freq,
+ "target_hist_counts": target_hist_counts,
+ "target_spectrum": target_spectrum,
+ "target_sf": target_sf,
+ "n_pixels": n_pixels,
+ }
+
+
+def run_matlab(matlab_bin: Path, runner: Path) -> None:
+ cmd = [str(matlab_bin), "-batch", f"run('{quote_matlab_path(runner)}')"]
+ completed = subprocess.run(cmd, text=True, capture_output=True)
+ if completed.stdout:
+ print(completed.stdout)
+ if completed.stderr:
+ print(completed.stderr, file=sys.stderr)
+ if completed.returncode != 0:
+ print(
+ f"MATLAB exited with status {completed.returncode}; "
+ "continuing so output checks can decide.",
+ file=sys.stderr,
+ )
+
+
+def run_python_processor(
+ *,
+ input_dir: Path,
+ output_root: Path,
+ implementation: str,
+ mode: int,
+ iterations: int,
+ seed: int,
+ target_hist: np.ndarray | None = None,
+ target_spectrum: np.ndarray | None = None,
+ keep_internal: bool = False,
+) -> tuple[Path, ImageProcessor]:
+ out_dir = output_root / "python" / implementation / f"mode_{mode}"
+ out_dir.mkdir(parents=True, exist_ok=True)
+ options = _build_options(
+ input_dir=input_dir,
+ output_dir=out_dir,
+ mode=mode,
+ iterations=iterations,
+ seed=seed,
+ target_hist=target_hist,
+ target_spectrum=target_spectrum,
+ legacy_mode=PYTHON_IMPLEMENTATIONS[implementation][1],
+ )
+ processor = ImageProcessor(
+ dataset=ImageDataset(options=options),
+ options=options,
+ from_unit_test=keep_internal,
+ )
+ if keep_internal:
+ processor.process()
+ processor.print_log_results()
+ if not getattr(processor.dataset.images, "has_list_array", False):
+ processor.dataset.save_images()
+ return out_dir, processor
+
+
+def read_saved_gray(path: Path, implementation: str = MATLAB_IMPLEMENTATION) -> np.ndarray:
+ image = np.asarray(Image.open(path))
+ if image.ndim == 2:
+ return image.astype(np.float64)
+
+ rgb = image[..., :3]
+ if implementation in {MATLAB_IMPLEMENTATION, LEGACY_IMPLEMENTATION}:
+ return MatlabOperators.uint8(
+ rgb2gray(rgb, weighting_standard="rec601", matlab_601=True)
+ ).astype(np.float64)
+
+ if implementation == MODERN_GRAY_IMPLEMENTATION:
+ if np.array_equal(rgb[..., 0], rgb[..., 1]) and np.array_equal(rgb[..., 0], rgb[..., 2]):
+ return rgb[..., 0].astype(np.float64)
+ return np.rint(
+ np.clip(rgb2gray(rgb, weighting_standard="rec601", matlab_601=False), 0, 255)
+ ).astype(np.float64)
+
+ raise ValueError(f"Unknown saved-output grayscale conversion for: {implementation}")
+
+
+def read_saved_target_image(path: Path, implementation: str = MATLAB_IMPLEMENTATION) -> np.ndarray:
+ if implementation != MODERN_GRAY_IMPLEMENTATION:
+ return read_saved_gray(path, implementation=implementation)
+
+ image = np.asarray(Image.open(path))
+ if image.ndim == 2:
+ image = np.dstack([image, image, image])
+ rgb = image[..., :3].astype(np.float64) / 255.0
+ converter = ColorConverter(rec_standard="rec601")
+ return converter.sRGB_to_xyY(rgb)[..., 2] * 255.0
+
+
+def load_shinier_initial_buffer(
+ *,
+ input_dir: Path,
+ output_root: Path,
+ seed: int,
+ full_tracking: bool,
+) -> dict[str, np.ndarray]:
+ probe_dir = output_root / "python" / "internal_input_probe"
+ probe_dir.mkdir(parents=True, exist_ok=True)
+ options = _build_options(
+ input_dir=input_dir,
+ output_dir=probe_dir,
+ mode=1,
+ iterations=1,
+ seed=seed,
+ )
+ processor = ImageProcessor(dataset=ImageDataset(options=options), options=options)
+ input_paths = sorted(input_dir.glob("*.png"))
+ internal = {
+ path.name: np.asarray(processor._initial_buffer[idx], dtype=np.float64)
+ for idx, path in enumerate(input_paths)
+ }
+ processor.dataset.close()
+
+ if full_tracking:
+ out_dir = output_root / "python" / "internal_input_gray"
+ out_dir.mkdir(parents=True, exist_ok=True)
+ for name, image in internal.items():
+ Image.fromarray(np.clip(np.rint(image), 0, 255).astype(np.uint8)).save(out_dir / name)
+ return internal
+
+
+def hist_l1(a: np.ndarray, b: np.ndarray) -> float:
+ """L1 distance between two image histograms (= 2 x total variation distance)."""
+ ha = np.bincount(a.astype(np.uint8).ravel(), minlength=256).astype(np.float64)
+ hb = np.bincount(b.astype(np.uint8).ravel(), minlength=256).astype(np.float64)
+ ha /= max(ha.sum(), 1.0)
+ hb /= max(hb.sum(), 1.0)
+ return 2.0 * compute_tvd_hist(ha, hb)
+
+
+def hist_l1_to_target(image: np.ndarray, target_hist_freq: np.ndarray) -> float:
+ """L1 distance between an image histogram and a target distribution."""
+ binned = np.clip(np.rint(image), 0, 255).astype(np.uint8)
+ hist = np.bincount(binned.ravel(), minlength=256).astype(np.float64)
+ hist /= max(hist.sum(), 1.0)
+ target = np.asarray(target_hist_freq, dtype=np.float64).reshape(-1)
+ target = target / target.sum()
+ return 2.0 * compute_tvd_hist(hist, target)
+
+
+def exported_hist_target(
+ target_hist_freq: np.ndarray,
+ *,
+ implementation: str,
+ n_pixels: int | None = None,
+) -> np.ndarray:
+ target = np.asarray(target_hist_freq, dtype=np.float64).reshape(-1)
+ target = target / target.sum()
+ if n_pixels is not None:
+ target = rounded_target_hist(target, int(n_pixels))
+
+ if implementation != MODERN_GRAY_IMPLEMENTATION:
+ return target
+
+ levels = np.arange(256, dtype=np.float64)
+ converter = ColorConverter(rec_standard="rec601")
+ srgb = converter.linRGB_to_sRGB(
+ np.dstack([levels / 255, levels / 255, levels / 255])
+ )[..., 0] * 255
+ saved = uint8_plus(srgb)
+ target_domain = converter.sRGB_to_xyY(
+ np.dstack([saved, saved, saved]).astype(np.float64) / 255
+ )[..., 2] * 255
+ mapped_bins = np.clip(np.rint(target_domain), 0, 255).astype(np.uint8).ravel()
+
+ mapped = np.zeros_like(target)
+ for src_bin, dst_bin in enumerate(mapped_bins):
+ mapped[int(dst_bin)] += target[src_bin]
+ mapped /= mapped.sum()
+ return mapped
+
+
+def spectrum_rmse_to_target(image: np.ndarray, target_spectrum: np.ndarray) -> float:
+ """RMSE between an image's Fourier magnitude spectrum and a target spectrum."""
+ magnitude, _ = image_spectrum(image / 255.0, rescale=False)
+ observed = magnitude.squeeze().astype(np.float64)
+ target = np.asarray(target_spectrum, dtype=np.float64).squeeze()
+ return float(compute_rmse(observed, target))
+
+
+def pre_range_spectrum_metrics(
+ *,
+ processor: ImageProcessor,
+ input_dir: Path,
+ mode: int,
+ targets: TargetSet,
+) -> dict[str, dict[str, float | str]]:
+ if mode not in SPECTRUM_MODES:
+ return {}
+
+ target = np.asarray(targets["target_spectrum"], dtype=np.float64)
+ if target.ndim == 2:
+ target = target[..., None]
+
+ rows: dict[str, dict[str, float | str]] = {}
+ for idx, input_path in enumerate(sorted(input_dir.glob("*.png"))):
+ phase = np.asarray(processor.dataset.phases[idx], dtype=np.float64)
+ if phase.ndim == 2:
+ phase = phase[..., None]
+
+ original_shape = np.asarray(processor._initial_buffer[idx]).shape[:2]
+ pre_range_channels: list[np.ndarray] = []
+ fourier_rmses: list[float] = []
+ for channel in range(target.shape[-1]):
+ xx, yy = pol2cart(target[:, :, channel], phase[:, :, channel])
+ reconstructed = np.real(np.fft.ifft2(np.fft.ifftshift(xx + yy * 1j)))
+ reconstructed_mag = np.abs(np.fft.fftshift(np.fft.fft2(reconstructed)))
+ fourier_rmses.append(float(compute_rmse(reconstructed_mag, target[:, :, channel])))
+ pre_range_channels.append(_crop_after_fft(reconstructed, original_shape))
+
+ pre_range01 = np.stack(pre_range_channels, axis=-1).squeeze()
+ pre_range255 = pre_range01 * 255.0
+ try:
+ pre_range_spec_rmse: float | str = spectrum_rmse_to_target(pre_range255, target)
+ except ValueError:
+ pre_range_spec_rmse = ""
+
+ if pre_range01.min() < 0.0 or pre_range01.max() > 1.0:
+ post_clip01 = soft_clip(
+ pre_range01,
+ min_value=0.0,
+ max_value=1.0,
+ max_percent=0.01,
+ verbose=False,
+ )
+ else:
+ post_clip01 = pre_range01
+ post_clip_spec_rmse = spectrum_rmse_to_target(post_clip01 * 255.0, target)
+
+ rows[input_path.name] = {
+ "pre_range_fourier_spectrum_rmse_to_python_target": float(np.mean(fourier_rmses)),
+ "pre_range_image_spectrum_rmse_to_python_target": pre_range_spec_rmse,
+ "post_soft_clip_spectrum_rmse_to_python_target": post_clip_spec_rmse,
+ "pre_range_below_zero_fraction": float(np.mean(pre_range01 < 0.0)),
+ "pre_range_above_one_fraction": float(np.mean(pre_range01 > 1.0)),
+ "pre_range_out_of_range_fraction": float(
+ np.mean((pre_range01 < 0.0) | (pre_range01 > 1.0))
+ ),
+ }
+ return rows
+
+
+def sf_rmse_to_target(image: np.ndarray, target_sf: np.ndarray, legacy_mode: bool = True) -> float:
+ """RMSE between an image's rotational-average SF profile and a target profile."""
+ magnitude, _ = image_spectrum(image / 255.0, rescale=False)
+ observed_mag = magnitude.squeeze().astype(np.float64)
+ radius = get_radius_grid(observed_mag.shape[0], observed_mag.shape[1], legacy_mode=legacy_mode)
+ observed_sf = rotational_avg(observed_mag, radius)
+ target = np.asarray(target_sf, dtype=np.float64).reshape(-1)
+ n = min(observed_sf.size, target.size)
+ return float(compute_rmse(observed_sf[:n], target[:n]))
+
+
+def pixel_difference_metrics(a: np.ndarray, b: np.ndarray) -> dict[str, object]:
+ diff = a - b
+ abs_diff = np.abs(diff)
+ return {
+ "rmse": float(compute_rmse(a, b)),
+ "mae": float(np.mean(abs_diff)),
+ "max_abs": float(abs_diff.max()),
+ "nonzero_pixels": int(np.count_nonzero(abs_diff)),
+ "equal_fraction": float(np.mean(abs_diff == 0)),
+ "hist_l1": hist_l1(a, b),
+ }
+
+
+def mean_value(rows: list[dict[str, object]], key: str) -> float:
+ return float(np.mean([float(row[key]) for row in rows]))
+
+
+def group_rows(rows: list[dict[str, object]], *keys: str):
+ groups: dict[tuple[object, ...], list[dict[str, object]]] = {}
+ for row in rows:
+ groups.setdefault(tuple(row.get(key, "") for key in keys), []).append(row)
+ return sorted(groups.items())
+
+
+def summarize_pixel_rows(rows: list[dict[str, object]]) -> dict[str, object]:
+ return {
+ "images": len(rows),
+ "mean_rmse": mean_value(rows, "rmse"),
+ "mean_mae": mean_value(rows, "mae"),
+ "max_abs": float(max(float(row["max_abs"]) for row in rows)),
+ "mean_equal_fraction": mean_value(rows, "equal_fraction"),
+ "mean_hist_l1": mean_value(rows, "hist_l1"),
+ }
+
+
+def target_row(
+ *,
+ mode: int,
+ implementation: str,
+ targets: TargetSet,
+ image_name: str,
+ metrics: dict[str, object],
+) -> dict[str, object]:
+ return {
+ "mode": mode,
+ "implementation": implementation,
+ "target": str(targets.get("target_name", "legacy")),
+ "image": image_name,
+ **{key: "" for key in TARGET_METRIC_KEYS},
+ **metrics,
+ }
+
+
+def target_metrics(
+ image: np.ndarray,
+ *,
+ mode: int,
+ targets: TargetSet,
+ prefix: str,
+ hist_target_freq: np.ndarray | None = None,
+) -> dict[str, object]:
+ metrics: dict[str, object] = {}
+ if mode in HIST_MODES:
+ metrics[f"{prefix}_hist_l1_to_python_target"] = hist_l1_to_target(
+ image,
+ hist_target_freq if hist_target_freq is not None else targets["target_hist_freq"],
+ )
+ if mode in SF_MODES:
+ metrics[f"{prefix}_sf_rmse_to_python_target"] = sf_rmse_to_target(
+ image,
+ targets["target_sf"],
+ legacy_mode=bool(targets.get("legacy_mode", True)),
+ )
+ if mode in SPECTRUM_MODES:
+ metrics[f"{prefix}_spectrum_rmse_to_python_target"] = spectrum_rmse_to_target(
+ image, targets["target_spectrum"]
+ )
+ return metrics
+
+
+def exported_target_metrics(
+ image: np.ndarray,
+ *,
+ implementation: str,
+ mode: int,
+ targets: TargetSet,
+) -> dict[str, object]:
+ hist_target_freq = None
+ if mode in HIST_MODES:
+ hist_target_freq = exported_hist_target(
+ targets["target_hist_freq"],
+ implementation=implementation,
+ n_pixels=int(targets["n_pixels"]),
+ )
+ return target_metrics(
+ image,
+ mode=mode,
+ targets=targets,
+ prefix="exported_png",
+ hist_target_freq=hist_target_freq,
+ )
+
+
+def compare_dirs(
+ matlab_dir: Path,
+ python_dir: Path,
+ mode: int,
+ implementation: str,
+) -> list[dict[str, object]]:
+ rows: list[dict[str, object]] = []
+ for matlab_path in sorted(matlab_dir.glob("*.png")):
+ python_path = python_dir / matlab_path.name
+ if not python_path.exists():
+ raise FileNotFoundError(f"Missing Python output: {python_path}")
+ m = read_saved_gray(matlab_path, implementation=MATLAB_IMPLEMENTATION)
+ p = read_saved_gray(python_path, implementation=implementation)
+ rows.append({
+ "mode": mode,
+ "reference": MATLAB_IMPLEMENTATION,
+ "python_output": implementation,
+ "image": matlab_path.name,
+ **pixel_difference_metrics(p, m),
+ })
+ return rows
+
+
+def compare_input_grayscale(
+ input_dir: Path,
+ output_root: Path,
+ *,
+ seed: int,
+ full_tracking: bool = False,
+) -> list[dict[str, object]]:
+ matlab_gray_dir = output_root / "matlab" / "input_gray"
+ if not matlab_gray_dir.exists():
+ return []
+ shinier_internal = load_shinier_initial_buffer(
+ input_dir=input_dir,
+ output_root=output_root,
+ seed=seed,
+ full_tracking=full_tracking,
+ )
+
+ rows: list[dict[str, object]] = []
+ for matlab_path in sorted(matlab_gray_dir.glob("*.png")):
+ if matlab_path.name not in shinier_internal:
+ raise FileNotFoundError(
+ f"Missing SHINIER internal input buffer for {matlab_path.name}"
+ )
+ m = read_saved_gray(matlab_path, implementation=MATLAB_IMPLEMENTATION)
+ p = shinier_internal[matlab_path.name]
+ rows.append({"image": matlab_path.name, **pixel_difference_metrics(p, m)})
+ write_csv(output_root / "input_grayscale_comparison.csv", rows)
+ return rows
+
+
+def aggregate(rows: list[dict[str, object]]) -> list[dict[str, object]]:
+ out = []
+ for (mode, reference, python_output), items in group_rows(
+ rows, "mode", "reference", "python_output"
+ ):
+ summary = summarize_pixel_rows(items)
+ out.append(
+ {
+ "mode": int(mode),
+ "reference": reference,
+ "python_output": python_output,
+ **{key: summary[key] for key in PIXEL_COLUMNS if key != "images"},
+ }
+ )
+ return out
+
+
+def compare_outputs_to_python_targets(
+ *,
+ output_dir: Path,
+ implementation: str,
+ mode: int,
+ targets: TargetSet,
+) -> list[dict[str, object]]:
+ if mode not in TARGET_MODES:
+ return []
+ rows: list[dict[str, object]] = []
+ for image_path in sorted(output_dir.glob("*.png")):
+ image = read_saved_target_image(image_path, implementation=implementation)
+ rows.append(
+ target_row(
+ mode=mode,
+ implementation=implementation,
+ targets=targets,
+ image_name=image_path.name,
+ metrics=exported_target_metrics(
+ image,
+ implementation=implementation,
+ mode=mode,
+ targets=targets,
+ ),
+ )
+ )
+ return rows
+
+
+def compare_processor_to_python_targets(
+ *,
+ processor: ImageProcessor,
+ input_dir: Path,
+ output_dir: Path,
+ implementation: str,
+ mode: int,
+ targets: TargetSet,
+) -> list[dict[str, object]]:
+ if mode not in TARGET_MODES:
+ return []
+ rows: list[dict[str, object]] = []
+ pre_range_by_image = pre_range_spectrum_metrics(
+ processor=processor,
+ input_dir=input_dir,
+ mode=mode,
+ targets=targets,
+ )
+ for idx, input_path in enumerate(sorted(input_dir.glob("*.png"))):
+ image = np.asarray(processor._final_buffer[idx]).squeeze().astype(np.float64)
+ metrics = {
+ **target_metrics(image, mode=mode, targets=targets, prefix="internal_buffer"),
+ **pre_range_by_image.get(input_path.name, {}),
+ }
+
+ saved_path = output_dir / input_path.name
+ if saved_path.exists():
+ exported_image = read_saved_target_image(saved_path, implementation=implementation)
+ metrics.update(
+ exported_target_metrics(
+ exported_image,
+ implementation=implementation,
+ mode=mode,
+ targets=targets,
+ )
+ )
+ rows.append(
+ target_row(
+ mode=mode,
+ implementation=implementation,
+ targets=targets,
+ image_name=input_path.name,
+ metrics=metrics,
+ )
+ )
+ return rows
+
+
+def aggregate_target_rows(rows: list[dict[str, object]]) -> list[dict[str, object]]:
+ out: list[dict[str, object]] = []
+ for (mode, implementation, target), items in group_rows(
+ rows, "mode", "implementation", "target"
+ ):
+ summary: dict[str, object] = {
+ "mode": int(mode),
+ "implementation": implementation,
+ "target": target,
+ }
+ for key in TARGET_METRIC_KEYS:
+ vals = [float(item[key]) for item in items if item[key] != ""]
+ summary[f"mean_{key}"] = float(np.mean(vals)) if vals else ""
+ out.append(summary)
+ return out
+
+
+def write_csv(path: Path, rows: list[dict[str, object]]) -> None:
+ if not rows:
+ return
+ path.parent.mkdir(parents=True, exist_ok=True)
+ with path.open("w", newline="") as f:
+ writer = csv.DictWriter(f, fieldnames=list(rows[0].keys()))
+ writer.writeheader()
+ writer.writerows(rows)
+
+
+def cleanup_tracking_artifacts(run_root: Path) -> None:
+ if not run_root.exists():
+ return
+ for path in run_root.iterdir():
+ if path.is_dir():
+ shutil.rmtree(path)
+ elif path.suffix.lower() != ".csv":
+ path.unlink()
+
+
+INTEGER_COLUMNS = {"mode", "images", "nonzero_pixels"}
+
+
+def format_table_value(column: str, value: object) -> str:
+ if value == "":
+ return "-"
+ if isinstance(value, bool):
+ return str(value).lower()
+ if isinstance(value, (int, np.integer)):
+ return str(int(value))
+ if column in INTEGER_COLUMNS and isinstance(value, (int, float, np.integer, np.floating)):
+ return str(int(value))
+ if isinstance(value, (int, float, np.integer, np.floating)):
+ return f"{float(value):.3e}"
+ return str(value)
+
+
+def print_table(
+ title: str,
+ rows: list[dict[str, object]],
+ columns: list[str],
+ *,
+ column_labels: dict[str, str] | None = None,
+ notes: list[str] | None = None,
+) -> None:
+ if not rows:
+ return
+ labels = column_labels or {}
+ headers = [labels.get(column, column) for column in columns]
+ formatted_rows = [
+ [format_table_value(column, row.get(column, "")) for column in columns]
+ for row in rows
+ ]
+ widths = [
+ max(len(headers[index]), *(len(row[index]) for row in formatted_rows))
+ for index, column in enumerate(columns)
+ ]
+ header = " | ".join(headers[index].ljust(widths[index]) for index, _ in enumerate(columns))
+ separator = "-+-".join("-" * width for width in widths)
+
+ print()
+ print(title)
+ print(separator)
+ print(header)
+ print(separator)
+ for row in formatted_rows:
+ print(" | ".join(value.ljust(widths[index]) for index, value in enumerate(row)))
+ print(separator)
+ if notes:
+ for note in notes:
+ print(note)
+
+
+def print_key_values(title: str, row: dict[str, object], keys: list[str]) -> None:
+ key_width = max(len(key) for key in keys)
+
+ print()
+ print(title)
+ print("-" * len(title))
+ for key in keys:
+ value = row.get(key, "")
+ if isinstance(value, bool):
+ text = str(value).lower()
+ else:
+ text = str(value)
+ print(f"{key.ljust(key_width)} : {text}")
+
+
+def prepare_run(args: argparse.Namespace) -> tuple[Path, Path, Path]:
+ run_root = (
+ args.run_root or args.output_dir / datetime.now().strftime("%Y%m%d_%H%M%S")
+ ).resolve()
+ run_root.mkdir(parents=True, exist_ok=True)
+ input_dir = run_root / "input"
+ if not input_dir.exists() or not list(input_dir.glob("*.png")):
+ copied_inputs = prepare_inputs(args.input_dir, run_root, args.limit)
+ input_dir = copied_inputs[0].parent
+
+ runner = write_matlab_runner(
+ output_root=run_root,
+ input_dir=input_dir,
+ shine_dir=args.shine_dir,
+ modes=args.modes,
+ iterations=args.iterations,
+ seed=args.seed,
+ use_python_targets=args.use_python_targets,
+ )
+ if not args.quiet_run_info:
+ print_key_values(
+ "Prepared diagnostic run",
+ {
+ "run_root": run_root,
+ "matlab_runner": runner,
+ "modes": " ".join(str(mode) for mode in args.modes),
+ "images": len(list(input_dir.glob("*.png"))),
+ "use_python_targets": args.use_python_targets,
+ "skip_matlab": args.skip_matlab,
+ "prepare_only": args.prepare_only,
+ },
+ RUN_INFO_KEYS,
+ )
+ return run_root, input_dir, runner
+
+
+def compute_target_sets(
+ *,
+ input_dir: Path,
+ run_root: Path,
+ seed: int,
+) -> TargetSets:
+ configs = [
+ ("legacy", True, True),
+ ("modern_gray", False, False),
+ ]
+ return {
+ name: compute_python_targets(
+ input_dir=input_dir,
+ output_root=run_root,
+ seed=seed,
+ legacy_mode=legacy_mode,
+ target_name=name,
+ export_matlab_targets=export_matlab_targets,
+ )
+ for name, legacy_mode, export_matlab_targets in configs
+ }
+
+
+def print_input_grayscale_results(rows: list[dict[str, object]], run_root: Path) -> None:
+ if rows:
+ print(f"input_gray_summary={run_root / 'input_grayscale_comparison.csv'}")
+ print_table(INPUT_GRAY_TITLE, [summarize_pixel_rows(rows)], PIXEL_COLUMNS)
+ return
+ print_table(
+ INPUT_GRAY_TITLE,
+ [
+ {
+ "status": "not_available",
+ "reason": "MATLAB input_gray outputs were not found",
+ }
+ ],
+ ["status", "reason"],
+ )
+
+
+def matlab_target_rows(
+ *,
+ run_root: Path,
+ mode: int,
+ skip_matlab: bool,
+ target_sets: TargetSets,
+) -> list[dict[str, object]]:
+ matlab_dir = run_root / "matlab" / f"mode_{mode}"
+ if skip_matlab and not matlab_dir.exists():
+ return []
+ if not matlab_dir.exists():
+ raise FileNotFoundError(f"Missing MATLAB outputs for mode {mode}: {matlab_dir}")
+ return compare_outputs_to_python_targets(
+ output_dir=matlab_dir,
+ implementation=MATLAB_IMPLEMENTATION,
+ mode=mode,
+ targets=target_sets["legacy"],
+ )
+
+
+def python_target_rows(
+ *,
+ args: argparse.Namespace,
+ input_dir: Path,
+ run_root: Path,
+ mode: int,
+ target_sets: TargetSets,
+) -> list[dict[str, object]]:
+ rows: list[dict[str, object]] = []
+ for implementation in PYTHON_IMPLEMENTATIONS:
+ targets = target_sets[PYTHON_IMPLEMENTATIONS[implementation][0]]
+ python_dir, processor = run_python_processor(
+ input_dir=input_dir,
+ output_root=run_root,
+ implementation=implementation,
+ mode=mode,
+ iterations=args.iterations,
+ seed=args.seed,
+ target_hist=np.asarray(targets["target_hist_freq"], dtype=np.float64),
+ target_spectrum=np.asarray(targets["target_spectrum"], dtype=np.float64),
+ keep_internal=True,
+ )
+ try:
+ rows.extend(
+ compare_processor_to_python_targets(
+ processor=processor,
+ input_dir=input_dir,
+ output_dir=python_dir,
+ implementation=implementation,
+ mode=mode,
+ targets=targets,
+ )
+ )
+ finally:
+ processor.dataset.close()
+ return rows
+
+
+def run_target_probe(
+ *,
+ args: argparse.Namespace,
+ input_dir: Path,
+ run_root: Path,
+ target_sets: TargetSets,
+) -> None:
+ target_detail_rows: list[dict[str, object]] = []
+ for mode in args.modes:
+ target_detail_rows.extend(
+ matlab_target_rows(
+ run_root=run_root,
+ mode=mode,
+ skip_matlab=args.skip_matlab,
+ target_sets=target_sets,
+ )
+ )
+ target_detail_rows.extend(
+ python_target_rows(
+ args=args,
+ input_dir=input_dir,
+ run_root=run_root,
+ mode=mode,
+ target_sets=target_sets,
+ )
+ )
+
+ target_summary_rows = aggregate_target_rows(target_detail_rows)
+ write_csv(run_root / "python_target_probe_detail.csv", target_detail_rows)
+ write_csv(run_root / "python_target_probe_summary.csv", target_summary_rows)
+ print(f"summary={run_root / 'python_target_probe_summary.csv'}")
+ print_table(
+ "MATLAB/Python vs implementation-specific fixed initial Python target",
+ target_summary_rows,
+ TARGET_TABLE_COLUMNS,
+ column_labels=TARGET_TABLE_LABELS,
+ notes=TARGET_TABLE_NOTES,
+ )
+
+
+def run_matlab_vs_python_probe(
+ *,
+ args: argparse.Namespace,
+ input_dir: Path,
+ run_root: Path,
+) -> None:
+ detail_rows: list[dict[str, object]] = []
+ for mode in args.modes:
+ matlab_dir = run_root / "matlab" / f"mode_{mode}"
+ if not matlab_dir.exists():
+ raise FileNotFoundError(f"Missing MATLAB outputs for mode {mode}: {matlab_dir}")
+ for implementation in PYTHON_IMPLEMENTATIONS:
+ python_dir, processor = run_python_processor(
+ input_dir=input_dir,
+ output_root=run_root,
+ implementation=implementation,
+ mode=mode,
+ iterations=args.iterations,
+ seed=args.seed,
+ )
+ processor.dataset.close()
+ detail_rows.extend(compare_dirs(matlab_dir, python_dir, mode, implementation))
+
+ summary_rows = aggregate(detail_rows)
+ write_csv(run_root / "matlab_shine_comparison_detail.csv", detail_rows)
+ write_csv(run_root / "matlab_shine_comparison_summary.csv", summary_rows)
+ print(f"summary={run_root / 'matlab_shine_comparison_summary.csv'}")
+ print_table(
+ "Python outputs relative to MATLAB SHINE",
+ summary_rows,
+ OUTPUT_COMPARISON_COLUMNS,
+ )
+
+
+def main() -> None:
+ args = parse_args()
+ run_root, input_dir, runner = prepare_run(args)
+
+ target_sets = None
+ if args.use_python_targets:
+ print(ANSI_BOLD_RED + "\n".join(f"WARNING: {line}" for line in TARGET_WARNING_LINES) + ANSI_RESET)
+ target_sets = compute_target_sets(
+ input_dir=input_dir,
+ run_root=run_root,
+ seed=args.seed,
+ )
+ if args.prepare_only:
+ return
+ if not args.skip_matlab:
+ run_matlab(args.matlab_bin, runner)
+
+ input_gray_rows = compare_input_grayscale(
+ input_dir,
+ run_root,
+ seed=args.seed,
+ full_tracking=args.full_tracking,
+ )
+ print_input_grayscale_results(input_gray_rows, run_root)
+
+ if args.use_python_targets:
+ assert target_sets is not None
+ run_target_probe(
+ args=args,
+ input_dir=input_dir,
+ run_root=run_root,
+ target_sets=target_sets,
+ )
+ else:
+ run_matlab_vs_python_probe(args=args, input_dir=input_dir, run_root=run_root)
+
+ if not args.full_tracking:
+ cleanup_tracking_artifacts(run_root)
+
+
+if __name__ == "__main__":
+ main()
diff --git a/tests/tools/run_matlab_shine_comparison.sh b/tests/tools/run_matlab_shine_comparison.sh
new file mode 100755
index 0000000..a86c32f
--- /dev/null
+++ b/tests/tools/run_matlab_shine_comparison.sh
@@ -0,0 +1,329 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+# Compare SHINIER (Python) against the original MATLAB SHINE toolbox.
+# Requires MATLAB and the SHINE toolbox:
+# http://www.mapageweb.umontreal.ca/gosselif/SHINE/
+#
+# Centralized paths and run settings. Override any of these from the shell, e.g.
+# RUN_ROOT=tmp/matlab_shine_comparison/my_run LIMIT=8 bash tests/tools/run_matlab_shine_comparison.sh
+# If MATLAB/Python/SHINE are not found, the script asks for the needed paths.
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+REPO_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
+
+PYTHON_BIN="${PYTHON_BIN:-python3}"
+MATLAB_BIN="${MATLAB_BIN:-}"
+SHINE_DIR="${SHINE_DIR:-}"
+
+OUTPUT_DIR="${OUTPUT_DIR:-$REPO_ROOT/tmp/matlab_shine_comparison}"
+RUN_NAME="${RUN_NAME:-full_comparison}"
+RUN_ROOT="${RUN_ROOT:-$OUTPUT_DIR/$RUN_NAME}"
+MATLAB_VS_PYTHON_ROOT="$RUN_ROOT/matlab_vs_python"
+PYTHON_TARGET_ROOT="$RUN_ROOT/python_targets"
+
+MODES="${MODES:-1 2 3 4 5 6 7 8}"
+LIMIT="${LIMIT:-8}"
+ITERATIONS="${ITERATIONS:-5}"
+SEED="${SEED:-42}"
+FULL_TRACKING="${FULL_TRACKING:-0}"
+
+PY_SCRIPT="$SCRIPT_DIR/matlab_shine_comparison.py"
+REQUIRED_SHINE_FILES=(lumMatch.m histMatch.m sfMatch.m specMatch.m)
+REQUIRED_SHINE_FUNCTIONS=(lumMatch histMatch sfMatch specMatch)
+
+read -r -a MODE_ARGS <<< "$MODES"
+
+BOLD="$(printf '\033[1m')"
+RESET="$(printf '\033[0m')"
+
+is_executable() {
+ local candidate="$1"
+ command -v "$candidate" >/dev/null 2>&1 || [[ -x "$candidate" ]]
+}
+
+detect_matlab_bin() {
+ local requested="$1"
+ local candidate
+ local found=""
+
+ if [[ -n "$requested" && "$requested" != "matlab" && -x "$requested" ]]; then
+ printf '%s' "$requested"
+ return
+ fi
+
+ if command -v matlab >/dev/null 2>&1; then
+ command -v matlab
+ return
+ fi
+
+ for candidate in \
+ /Applications/MATLAB_R*.app/bin/matlab \
+ /Applications/MATLAB.app/bin/matlab \
+ /usr/local/MATLAB/R*/bin/matlab \
+ /opt/MATLAB/R*/bin/matlab \
+ /usr/local/bin/matlab \
+ /usr/bin/matlab \
+ "/c/Program Files/MATLAB"/R*/bin/matlab.exe \
+ "/mnt/c/Program Files/MATLAB"/R*/bin/matlab.exe
+ do
+ if [[ -x "$candidate" ]]; then
+ found="$candidate"
+ fi
+ done
+
+ printf '%s' "$found"
+}
+
+print_path_help() {
+ local label="$1"
+
+ case "$label" in
+ Python)
+ printf '\n%sPython executable path%s\n' "$BOLD" "$RESET" >&2
+ printf ' What to provide: the Python executable used to run this diagnostic.\n' >&2
+ printf ' Examples:\n' >&2
+ printf ' - python3\n' >&2
+ printf ' - .venv/bin/python\n' >&2
+ printf ' - /Users/you/environments/shinier_venv/bin/python\n' >&2
+ ;;
+ MATLAB)
+ printf '\n%sMATLAB executable path%s\n' "$BOLD" "$RESET" >&2
+ printf ' What to provide: the MATLAB executable, not the MATLAB.app folder.\n' >&2
+ printf ' Examples:\n' >&2
+ printf ' - /Applications/MATLAB_R2025a.app/bin/matlab\n' >&2
+ printf ' - /usr/local/MATLAB/R2025a/bin/matlab\n' >&2
+ printf ' - C:\\Program Files\\MATLAB\\R2025a\\bin\\matlab.exe\n' >&2
+ ;;
+ SHINE)
+ printf '\n%sSHINE toolbox folder%s\n' "$BOLD" "$RESET" >&2
+ printf ' What to provide: the folder containing MATLAB files such as:\n' >&2
+ printf ' - lumMatch.m\n' >&2
+ printf ' - histMatch.m\n' >&2
+ printf ' - sfMatch.m\n' >&2
+ printf ' - specMatch.m\n' >&2
+ printf ' Example:\n' >&2
+ printf ' - /Users/you/projects/shinetoolbox\n' >&2
+ printf ' Download SHINE:\n' >&2
+ printf ' - http://www.mapageweb.umontreal.ca/gosselif/SHINE/\n' >&2
+ printf ' Press Enter only if MATLAB can already find those SHINE functions.\n' >&2
+ ;;
+ esac
+}
+
+prompt_required_path() {
+ local label="$1"
+ local current="$2"
+ local env_var="$3"
+ local value="$current"
+
+ while ! is_executable "$value"; do
+ printf '%s not found: %s\n' "$label" "$value" >&2
+ if [[ ! -t 0 ]]; then
+ print_path_help "$label"
+ printf 'Set %s=/path/to/executable or run this script interactively.\n' "$env_var" >&2
+ exit 1
+ fi
+ print_path_help "$label"
+ printf '\nEnter %s executable path: ' "$label" >&2
+ read -r value
+ if [[ -z "$value" ]]; then
+ printf '%s path is required.\n' "$label" >&2
+ exit 1
+ fi
+ done
+
+ printf '%s' "$value"
+}
+
+prompt_optional_dir() {
+ local label="$1"
+ local current="$2"
+ local env_var="$3"
+ local value="$current"
+
+ while [[ -n "$value" && ! -d "$value" ]]; do
+ printf '%s directory not found: %s\n' "$label" "$value" >&2
+ if [[ ! -t 0 ]]; then
+ print_path_help "$label"
+ printf 'Set %s=/path/to/directory or leave it unset if MATLAB already has it on path.\n' "$env_var" >&2
+ exit 1
+ fi
+ print_path_help "$label"
+ printf '\nEnter %s directory path, or press Enter if MATLAB already has it on path: ' "$label" >&2
+ read -r value
+ done
+
+ if [[ -z "$value" && -t 0 && -z "${SHINE_DIR:-}" ]]; then
+ print_path_help "$label"
+ printf '\nEnter %s directory path, or press Enter if MATLAB already has it on path: ' "$label" >&2
+ read -r value
+ while [[ -n "$value" && ! -d "$value" ]]; do
+ printf '%s directory not found: %s\n' "$label" "$value" >&2
+ print_path_help "$label"
+ printf '\nEnter %s directory path, or press Enter if MATLAB already has it on path: ' "$label" >&2
+ read -r value
+ done
+ fi
+
+ printf '%s' "$value"
+}
+
+validate_shine_dir() {
+ local missing=()
+ local file
+
+ for file in "${REQUIRED_SHINE_FILES[@]}"; do
+ if [[ ! -f "$SHINE_DIR/$file" ]]; then
+ missing+=("$file")
+ fi
+ done
+
+ if [[ "${#missing[@]}" -gt 0 ]]; then
+ printf '\n%sSHINE toolbox check failed%s\n' "$BOLD" "$RESET" >&2
+ printf ' Folder checked:\n' >&2
+ printf ' - %s\n' "$SHINE_DIR" >&2
+ printf ' Missing required MATLAB files:\n' >&2
+ for file in "${missing[@]}"; do
+ printf ' - %s\n' "$file" >&2
+ done
+ printf ' Download SHINE:\n' >&2
+ printf ' - http://www.mapageweb.umontreal.ca/gosselif/SHINE/\n' >&2
+ printf ' Then rerun with SHINE_DIR=/path/to/shinetoolbox.\n' >&2
+ exit 1
+ fi
+}
+
+validate_shine_matlab_path() {
+ local matlab_code
+ local function_list
+
+ function_list="$(printf "'%s'," "${REQUIRED_SHINE_FUNCTIONS[@]}")"
+ function_list="${function_list%,}"
+ matlab_code="funcs={${function_list}}; missing={}; for k=1:numel(funcs), if exist(funcs{k}, 'file') ~= 2, missing{end+1}=funcs{k}; end; end; if ~isempty(missing), fprintf(2, '\\nSHINE toolbox check failed\\n'); fprintf(2, ' MATLAB cannot find required SHINE functions:\\n'); for k=1:numel(missing), fprintf(2, ' - %s\\n', missing{k}); end; fprintf(2, ' Provide SHINE_DIR=/path/to/shinetoolbox or download SHINE from:\\n'); fprintf(2, ' - http://www.mapageweb.umontreal.ca/gosselif/SHINE/\\n'); exit(1); end"
+
+ if ! "$MATLAB_BIN" -batch "$matlab_code"; then
+ printf '\n%sSHINE toolbox check failed%s\n' "$BOLD" "$RESET" >&2
+ printf ' MATLAB could not confirm that SHINE is on the MATLAB path.\n' >&2
+ printf ' Provide SHINE_DIR=/path/to/shinetoolbox and rerun.\n' >&2
+ exit 1
+ fi
+}
+
+validate_shine_setup() {
+ printf '\n[preflight] Checking SHINE toolbox availability\n'
+ if [[ -n "$SHINE_DIR" ]]; then
+ validate_shine_dir
+ else
+ validate_shine_matlab_path
+ fi
+}
+
+run_matlab_runner() {
+ local runner="$1"
+ local matlab_runner="${runner//\'/\'\'}"
+ "$MATLAB_BIN" -batch "run('$matlab_runner')"
+}
+
+cleanup_csv_only_root() {
+ local root="$1"
+ local path
+
+ [[ -d "$root" ]] || return 0
+ find "$root" -mindepth 1 -maxdepth 1 -type d -exec rm -rf {} +
+ while IFS= read -r -d '' path; do
+ case "$path" in
+ *.csv) ;;
+ *) rm -f "$path" ;;
+ esac
+ done < <(find "$root" -mindepth 1 -maxdepth 1 -type f -print0)
+}
+
+cleanup_default_artifacts() {
+ if [[ "$FULL_TRACKING" == "1" || "$FULL_TRACKING" == "true" || "$FULL_TRACKING" == "yes" ]]; then
+ return 0
+ fi
+ cleanup_csv_only_root "$MATLAB_VS_PYTHON_ROOT"
+ cleanup_csv_only_root "$PYTHON_TARGET_ROOT"
+}
+
+print_config() {
+ printf '\nMATLAB SHINE vs SHINIER comparison\n'
+ printf '%-24s %s\n' 'repo_root' "$REPO_ROOT"
+ printf '%-24s %s\n' 'python_bin' "$PYTHON_BIN"
+ printf '%-24s %s\n' 'matlab_bin' "$MATLAB_BIN"
+ printf '%-24s %s\n' 'shine_dir' "${SHINE_DIR:-MATLAB path}"
+ printf '%-24s %s\n' 'run_root' "$RUN_ROOT"
+ printf '%-24s %s\n' 'modes' "$MODES"
+ printf '%-24s %s\n' 'limit' "$LIMIT"
+ printf '%-24s %s\n' 'iterations' "$ITERATIONS"
+ printf '%-24s %s\n' 'full_tracking' "$FULL_TRACKING"
+ printf '%-24s %s\n\n' 'seed' "$SEED"
+}
+
+PYTHON_BIN="$(prompt_required_path Python "$PYTHON_BIN" PYTHON_BIN)"
+MATLAB_BIN="$(detect_matlab_bin "$MATLAB_BIN")"
+MATLAB_BIN="$(prompt_required_path MATLAB "$MATLAB_BIN" MATLAB_BIN)"
+SHINE_DIR="$(prompt_optional_dir SHINE "$SHINE_DIR" SHINE_DIR)"
+
+COMMON_ARGS=(
+ --modes "${MODE_ARGS[@]}"
+ --limit "$LIMIT"
+ --iterations "$ITERATIONS"
+ --seed "$SEED"
+ --matlab-bin "$MATLAB_BIN"
+ --quiet-run-info
+)
+
+if [[ -n "$SHINE_DIR" ]]; then
+ COMMON_ARGS+=(--shine-dir "$SHINE_DIR")
+fi
+
+if [[ "$FULL_TRACKING" == "1" || "$FULL_TRACKING" == "true" || "$FULL_TRACKING" == "yes" ]]; then
+ COMMON_ARGS+=(--full-tracking)
+fi
+
+cd "$REPO_ROOT"
+mkdir -p "$RUN_ROOT"
+trap cleanup_default_artifacts EXIT
+print_config
+validate_shine_setup
+
+printf '\n[1/6] Prepare MATLAB vs Python run\n'
+"$PYTHON_BIN" "$PY_SCRIPT" "${COMMON_ARGS[@]}" \
+ --prepare-only \
+ --run-root "$MATLAB_VS_PYTHON_ROOT"
+
+printf '\n[2/6] Run MATLAB for MATLAB vs Python outputs\n'
+run_matlab_runner "$MATLAB_VS_PYTHON_ROOT/run_matlab_shine.m"
+
+printf '\n[3/6] Compare MATLAB vs Python outputs\n'
+"$PYTHON_BIN" "$PY_SCRIPT" "${COMMON_ARGS[@]}" \
+ --skip-matlab \
+ --run-root "$MATLAB_VS_PYTHON_ROOT"
+
+printf '\n[4/6] Prepare fixed Python target run\n'
+"$PYTHON_BIN" "$PY_SCRIPT" "${COMMON_ARGS[@]}" \
+ --use-python-targets \
+ --prepare-only \
+ --run-root "$PYTHON_TARGET_ROOT"
+
+printf '\n[5/6] Run MATLAB with fixed Python targets\n'
+run_matlab_runner "$PYTHON_TARGET_ROOT/run_matlab_shine.m"
+
+printf '\n[6/6] Compare MATLAB/Python against fixed Python targets\n'
+"$PYTHON_BIN" "$PY_SCRIPT" "${COMMON_ARGS[@]}" \
+ --use-python-targets \
+ --skip-matlab \
+ --run-root "$PYTHON_TARGET_ROOT"
+
+printf '\nDone. Main outputs:\n'
+printf '%-36s %s\n' 'MATLAB vs Python summary' "$MATLAB_VS_PYTHON_ROOT/matlab_shine_comparison_summary.csv"
+printf '%-36s %s\n' 'MATLAB vs Python detail' "$MATLAB_VS_PYTHON_ROOT/matlab_shine_comparison_detail.csv"
+printf '%-36s %s\n' 'Python target summary' "$PYTHON_TARGET_ROOT/python_target_probe_summary.csv"
+printf '%-36s %s\n' 'Python target detail' "$PYTHON_TARGET_ROOT/python_target_probe_detail.csv"
+printf '%-36s %s\n' 'Input grayscale comparison' "$MATLAB_VS_PYTHON_ROOT/input_grayscale_comparison.csv"
+if [[ "$FULL_TRACKING" == "1" || "$FULL_TRACKING" == "true" || "$FULL_TRACKING" == "yes" ]]; then
+ printf '%-36s %s\n' 'Full tracking artifacts' "$RUN_ROOT"
+else
+ printf '\nIntermediate images/scripts were removed. Set FULL_TRACKING=1 to keep them.\n'
+fi
From 815abb5dd878c7b62446da4b8b25a938541af6e5 Mon Sep 17 00:00:00 2001
From: Kaapra
Date: Thu, 6 Aug 2026 14:01:55 -0400
Subject: [PATCH 4/7] Bumping version
---
.github/workflows/tests.yml | 4 ++--
README.md | 2 +-
documentation/documentation.md | 2 +-
documentation/readthedocs/conf.py | 2 +-
pyproject.toml | 2 +-
src/shinier/__init__.py | 4 ++--
6 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml
index 30ec5f0..b2a64c8 100644
--- a/.github/workflows/tests.yml
+++ b/.github/workflows/tests.yml
@@ -2,9 +2,9 @@ name: Tests
on:
push:
- branches: [ main, dev_1.4, dev_1.7, dev_v0.2.0, dev_v0.2.1]
+ branches: [ main, dev_1.4, dev_1.7, dev_v0.2.0, dev_v0.2.1, dev_v0.2.2 ]
pull_request:
- branches: [ main, dev_1.4, dev_1.7, dev_v0.2.0, dev_v0.2.1 ]
+ branches: [ main, dev_1.4, dev_1.7, dev_v0.2.0, dev_v0.2.1, dev_v0.2.2 ]
jobs:
tests:
diff --git a/README.md b/README.md
index e57339e..f5b9101 100644
--- a/README.md
+++ b/README.md
@@ -152,5 +152,5 @@ See [LICENSE](LICENSE) for more information.
---
Code developed by Nicolas Dupuis-Roy and Mathias Salvas-Hébert
- Version 0.2.0 - Complete technical documentation
+ Version 0.2.2 - Complete technical documentation
diff --git a/documentation/documentation.md b/documentation/documentation.md
index 7c1f46f..c70f49a 100644
--- a/documentation/documentation.md
+++ b/documentation/documentation.md
@@ -937,5 +937,5 @@ Composite modes (5-8) apply **two sequential transformations** (e.g., spectrum m
Code developed by Nicolas Dupuis-Roy and Mathias Salvas-Hébert
- Version 0.2.0 - Complete technical documentation
+ Version 0.2.2 - Complete technical documentation
diff --git a/documentation/readthedocs/conf.py b/documentation/readthedocs/conf.py
index f91a09c..e929630 100644
--- a/documentation/readthedocs/conf.py
+++ b/documentation/readthedocs/conf.py
@@ -16,7 +16,7 @@
try:
from shinier import __version__
except Exception:
- __version__ = "0.2.0"
+ __version__ = "0.2.2"
version = __version__
release = __version__
diff --git a/pyproject.toml b/pyproject.toml
index b02dfcf..2e8edb9 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "shinier"
-version = "0.2.0"
+version = "0.2.2"
description = "Python port of the SHINE toolbox with added options (color management, dithering, EHS), optimized for large image sets."
readme = "README.md"
license = "BSD-3-Clause"
diff --git a/src/shinier/__init__.py b/src/shinier/__init__.py
index ae8c30a..bb0a853 100644
--- a/src/shinier/__init__.py
+++ b/src/shinier/__init__.py
@@ -15,8 +15,8 @@
"""
# Metadata
-__author__ = "Nicolas Dupuis-Roy"
-__version__ = "0.2.0"
+__author__ = "Nicolas Dupuis-Roy and Mathias Salvas-Hebert"
+__version__ = "0.2.2"
__email__ = "nicolas.dupuis.roy@umontreal.ca"
# For direct importation
From e4b8f7feb1642e4131cc20ef280cc7695adf1f8d Mon Sep 17 00:00:00 2001
From: Kaapra
Date: Thu, 6 Aug 2026 17:31:47 -0400
Subject: [PATCH 5/7] -Support Python 3.13, 3.14 -Fix issue with readthedocs
-Update min dependencies -Better handling of cconvolve
---
.github/workflows/tests.yml | 2 +-
documentation/contributing.md | 2 +-
pyproject.toml | 10 +--
src/shinier/__init__.py | 35 +++++++++--
src/shinier/color/GamutControl.py | 10 +--
src/shinier/color/chroma_eval.py | 14 ++---
src/shinier/utils.py | 8 +--
.../test_install_all_pythons_on_linux.sh | 2 +
tests/tools/matlab_shine_comparison.py | 2 +-
tests/unit_tests/OptionalCconvolve_test.py | 61 +++++++++++++++++++
10 files changed, 116 insertions(+), 30 deletions(-)
create mode 100644 tests/unit_tests/OptionalCconvolve_test.py
diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml
index b2a64c8..0746ed2 100644
--- a/.github/workflows/tests.yml
+++ b/.github/workflows/tests.yml
@@ -15,7 +15,7 @@ jobs:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
- python-version: ["3.9", "3.10", "3.11", "3.12"]
+ python-version: ["3.9", "3.10", "3.11", "3.12", "3.13", "3.14"]
steps:
# 1) Récupérer le code
diff --git a/documentation/contributing.md b/documentation/contributing.md
index a89a82a..471fca0 100644
--- a/documentation/contributing.md
+++ b/documentation/contributing.md
@@ -40,7 +40,7 @@ By participating, you agree to uphold a standard of professional, inclusive, and
## Development Setup
-> **Python:** >=3.9, <3.13
+> **Python:** >=3.9, <3.15
> **OS:** macOS / Linux / Windows
> **Optional:** C/C++ toolchain for the Cython-compiled `_cconvolve` extension (speeds up convolution)
diff --git a/pyproject.toml b/pyproject.toml
index 2e8edb9..b3b9040 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,5 +1,5 @@
[build-system]
-requires = ["setuptools>=77", "wheel", "numpy>=1.22"]
+requires = ["setuptools>=77", "wheel", "numpy>=2.0"]
build-backend = "setuptools.build_meta"
[project]
@@ -13,12 +13,11 @@ authors = [
{ name = "Nicolas Dupuis-Roy", email = "nicolas.dupuis.roy@umontreal.ca" },
{ name = "Mathias Salvas-Hébert", email = "mathias.salvas-hebert@umontreal.ca" }
]
-requires-python = ">=3.9,<3.13"
+requires-python = ">=3.9,<3.15"
dependencies = [
- "numpy>=1.22,<2.1",
+ "numpy>=1.23",
"Pillow>=9.0.0",
"tqdm>=4.60",
- "cython>=3.0",
"matplotlib>=3.9.0",
"pydantic>=2.12",
]
@@ -27,6 +26,8 @@ classifiers = [
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
+ "Programming Language :: Python :: 3.13",
+ "Programming Language :: Python :: 3.14",
"Operating System :: OS Independent",
]
@@ -38,6 +39,7 @@ dev = [
"pytest>=8.3.0",
"wheel>=0.44.0",
"setuptools>=77",
+ "cython>=3.0",
"scikit-image>=0.24.0",
"colour-science >= 0.4.0",
"build>=1.2.1",
diff --git a/src/shinier/__init__.py b/src/shinier/__init__.py
index bb0a853..c1a0735 100644
--- a/src/shinier/__init__.py
+++ b/src/shinier/__init__.py
@@ -22,17 +22,40 @@
# For direct importation
from importlib import util
from pathlib import Path
-_HAS_CYTHON = util.find_spec("shinier._cconvolve") is not None
+import sys
+import warnings
+
+_HAS_CYTHON = False
+convolve2d_direct = None
+convolve2d_separable = None
# This is the *package* root: src/shinier in dev, site-packages/shinier when installed
DEV_ROOT = Path(__file__).resolve().parents[2]
REPO_ROOT = Path(__file__).resolve().parent
-if _HAS_CYTHON:
- from ._cconvolve import convolve2d_direct, convolve2d_separable
-else:
- convolve2d_direct = None
- convolve2d_separable = None
+if util.find_spec("shinier._cconvolve") is not None:
+ try:
+ from ._cconvolve import convolve2d_direct, convolve2d_separable
+ _HAS_CYTHON = True
+ except Exception as exc:
+ try:
+ import numpy as _np
+
+ numpy_version = _np.__version__
+ except Exception:
+ numpy_version = "unavailable"
+
+ warnings.warn(
+ "SHINIER could not load the optional compiled convolution extension "
+ "(`shinier._cconvolve`). SHINIER will keep working, but convolution-heavy "
+ "operations will use the slower NumPy fallback. "
+ f"Python: {sys.version.split()[0]}; NumPy: {numpy_version}. "
+ "If you want the faster compiled extension, try reinstalling SHINIER after "
+ "upgrading pip, setuptools, wheel, and NumPy, and make sure a C++ compiler "
+ f"is available. Original error: {exc!r}",
+ RuntimeWarning,
+ stacklevel=2,
+ )
__all__ = [
"Options",
diff --git a/src/shinier/color/GamutControl.py b/src/shinier/color/GamutControl.py
index 5697e28..b4df93a 100644
--- a/src/shinier/color/GamutControl.py
+++ b/src/shinier/color/GamutControl.py
@@ -176,7 +176,7 @@ def _reliability_from_Y01(Y01: np.ndarray, fade_width: float = 0.05) -> np.ndarr
return np.clip(dist_to_edge / fade_width, 0.0, 1.0)
@staticmethod
- def _min_or_quantile(values: np.ndarray, quantile_threshold: float | None) -> float:
+ def _min_or_quantile(values: np.ndarray, quantile_threshold: Optional[float]) -> float:
"""Return min(values) or a lower-quantile if configured."""
if values.size == 0:
return 1.0
@@ -185,7 +185,7 @@ def _min_or_quantile(values: np.ndarray, quantile_threshold: float | None) -> fl
return float(np.quantile(values, quantile_threshold))
@staticmethod
- def _format_quantile_msg(quantile_threshold: float | None) -> str:
+ def _format_quantile_msg(quantile_threshold: Optional[float]) -> str:
"""Format a short message describing quantile clipping."""
if quantile_threshold is None:
return ''
@@ -194,9 +194,9 @@ def _format_quantile_msg(quantile_threshold: float | None) -> str:
def _log_image_overflow(
self,
kind: str,
- idx: int | None,
+ idx: Optional[int],
local_min: float,
- quantile_threshold: float | None,
+ quantile_threshold: Optional[float],
verbose: bool,
) -> None:
"""Log a standardized per-image overflow message.
@@ -277,7 +277,7 @@ def apply_low_Y_desaturation(
self,
Y: np.ndarray,
other: np.ndarray,
- idx: int | None = None,
+ idx: Optional[int] = None,
verbose: bool = False) -> Tuple[np.ndarray, np.ndarray]:
"""[S1] Optionally desaturate chroma for low-luminance pixels.
diff --git a/src/shinier/color/chroma_eval.py b/src/shinier/color/chroma_eval.py
index 42f1b7a..9205334 100644
--- a/src/shinier/color/chroma_eval.py
+++ b/src/shinier/color/chroma_eval.py
@@ -197,15 +197,13 @@ def mean_chroma_loss_pct_lab(
"""Compute the global relative mean chroma loss percentage in CIELAB.
Both sRGB images are converted to CIE Lab, and chroma is computed for each
- pixel as ``C* = sqrt(a*^2 + b*^2)``..
+ pixel as ``C* = sqrt(a*^2 + b*^2)``.
- The relative mean chroma loss percentage is defined as::
-
- ``100 * (E[C*_before] - E[C*_after]) / E[C*_before]``
-
- where ``E`` denotes the mean across all image pixels. A positive percentage
- indicates a reduction in mean chroma, whereas a negative percentage indicates
- an increase.
+ The relative mean chroma loss percentage is defined as
+ ``100 * (E[C*_before] - E[C*_after]) / E[C*_before]``, where ``E`` denotes
+ the mean across all image pixels. A positive percentage indicates a
+ reduction in mean chroma, whereas a negative percentage indicates an
+ increase.
Parameters
----------
diff --git a/src/shinier/utils.py b/src/shinier/utils.py
index c33cadc..a3b707b 100644
--- a/src/shinier/utils.py
+++ b/src/shinier/utils.py
@@ -320,16 +320,16 @@ class StimulusMasker:
>>> final_mask = masker.interactive_mask(image)
"""
- image_size: int | tuple[int, int]
+ image_size: Union[int, Tuple[int, int]]
cutoff_a: float
- cutoff_b: float | None = None
+ cutoff_b: Optional[float] = None
offset_a: float = 0.0
offset_b: float = 0.0
mask_type: MaskType = "feathered_disk"
sigma: float = 2.0
edge_width: float = 2.0
background: float = 0.5
- output_dtype: np.dtype | type = np.float64
+ output_dtype: Union[np.dtype, type] = np.float64
def mask(self) -> np.ndarray:
"""Generate mask as float64 in [0, 1]."""
@@ -2846,7 +2846,7 @@ def print_log(logs: List[str], log_path: Union[Path, str], log_name: Optional[st
filename = Path(log_path) / log_name
# Write each log to a new line in the file
- with open(filename, 'w') as file:
+ with open(filename, 'w', encoding='utf-8') as file:
for log in logs:
file.write(strip_ansi(log) + '\n')
diff --git a/tests/other_tests/test_install_all_pythons_on_linux.sh b/tests/other_tests/test_install_all_pythons_on_linux.sh
index 5380ad3..a2757d4 100644
--- a/tests/other_tests/test_install_all_pythons_on_linux.sh
+++ b/tests/other_tests/test_install_all_pythons_on_linux.sh
@@ -24,6 +24,8 @@ IMAGES=(
"python:3.10-slim-bookworm"
"python:3.11-slim-bookworm"
"python:3.12-slim-bookworm"
+ "python:3.13-slim-bookworm"
+ "python:3.14-slim-bookworm"
)
ok=()
diff --git a/tests/tools/matlab_shine_comparison.py b/tests/tools/matlab_shine_comparison.py
index a1624d5..f8184cd 100644
--- a/tests/tools/matlab_shine_comparison.py
+++ b/tests/tools/matlab_shine_comparison.py
@@ -957,7 +957,7 @@ def write_csv(path: Path, rows: list[dict[str, object]]) -> None:
if not rows:
return
path.parent.mkdir(parents=True, exist_ok=True)
- with path.open("w", newline="") as f:
+ with path.open("w", newline="", encoding="utf-8") as f:
writer = csv.DictWriter(f, fieldnames=list(rows[0].keys()))
writer.writeheader()
writer.writerows(rows)
diff --git a/tests/unit_tests/OptionalCconvolve_test.py b/tests/unit_tests/OptionalCconvolve_test.py
new file mode 100644
index 0000000..6c87319
--- /dev/null
+++ b/tests/unit_tests/OptionalCconvolve_test.py
@@ -0,0 +1,61 @@
+import subprocess
+import sys
+import textwrap
+
+import pytest
+
+
+pytestmark = pytest.mark.unit_tests
+
+
+def test_import_falls_back_when_optional_cconvolve_fails():
+ code = textwrap.dedent(
+ """
+ import importlib.abc
+ import importlib.machinery
+ import sys
+ import warnings
+
+ class BrokenCconvolveFinder(importlib.abc.MetaPathFinder):
+ def find_spec(self, fullname, path=None, target=None):
+ if fullname == "shinier._cconvolve":
+ return importlib.machinery.ModuleSpec(fullname, BrokenCconvolveLoader())
+ return None
+
+ class BrokenCconvolveLoader(importlib.abc.Loader):
+ def create_module(self, spec):
+ return None
+
+ def exec_module(self, module):
+ raise ImportError("simulated NumPy ABI mismatch")
+
+ sys.meta_path.insert(0, BrokenCconvolveFinder())
+
+ with warnings.catch_warnings(record=True) as caught:
+ warnings.simplefilter("always")
+ import shinier
+
+ assert shinier._HAS_CYTHON is False
+ assert shinier.convolve2d_direct is None
+ assert shinier.convolve2d_separable is None
+ assert any(
+ "optional compiled convolution extension" in str(warning.message)
+ and "slower NumPy fallback" in str(warning.message)
+ and "Python:" in str(warning.message)
+ and "NumPy:" in str(warning.message)
+ and "reinstalling SHINIER" in str(warning.message)
+ and "simulated NumPy ABI mismatch" in str(warning.message)
+ for warning in caught
+ )
+ """
+ )
+
+ result = subprocess.run(
+ [sys.executable, "-W", "always", "-c", code],
+ stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE,
+ text=True,
+ check=False,
+ )
+
+ assert result.returncode == 0, result.stdout + result.stderr
From 0e69c0479e491d43b86f04753fc6506e6421266b Mon Sep 17 00:00:00 2001
From: Kaapra
Date: Fri, 7 Aug 2026 11:04:55 -0400
Subject: [PATCH 6/7] -Update the references to the paper. -Add GIF for
StimulusMasker -Clean the format of the documentation -Add a PDF of the paper
---
README.md | 39 +++---
documentation/contributing.md | 2 +-
documentation/demos.md | 30 ++---
documentation/documentation.md | 123 ++++++++++--------
documentation/figures/sliding_puzzle.png | Bin 0 -> 41457 bytes
.../_static/dynamic_stim_masker.gif | Bin 0 -> 2139368 bytes
documentation/readthedocs/_static/shinier.pdf | Bin 0 -> 1510856 bytes
documentation/readthedocs/index.md | 1 +
documentation/readthedocs/paper.md | 20 +++
documentation/readthedocs/project-links.md | 1 +
src/shinier/__init__.py | 5 +-
tests/README.md | 32 ++---
12 files changed, 146 insertions(+), 107 deletions(-)
create mode 100644 documentation/figures/sliding_puzzle.png
create mode 100644 documentation/readthedocs/_static/dynamic_stim_masker.gif
create mode 100644 documentation/readthedocs/_static/shinier.pdf
create mode 100644 documentation/readthedocs/paper.md
diff --git a/README.md b/README.md
index f5b9101..8428608 100644
--- a/README.md
+++ b/README.md
@@ -1,4 +1,4 @@
-# 🌟 SHINIER
+# SHINIER
```text
███████╗██╗ ██╗██╗███╗ ██╗██╗███████╗██████╗
██╔════╝██║ ██║██║████╗ ██║██║██╔════╝██╔══██╗
@@ -15,26 +15,26 @@
[](https://github.com/Charestlab/shinier/actions/workflows/tests.yml)
---
-## 🎯 Overview
+## Overview
SHINIER is a modern Python implementation of SHINE (Spectrum, Histogram, and Intensity Normalization and Equalization), originally developed in MATLAB by Willenbockel et al., 2010. It provides precise control over luminance, contrast, histograms, and spectral content across large image sets for well-calibrated visual experiments.
### Key Features and Improvements
-- 🎨 **Color Processing** — New modes for color image control with modern color-space standards (Rec.601 / Rec.709 / Rec.2020).
-- 🖼️ **Dithering Support** — Reduces quantization artifacts and enhances output image quality.
-- ⚡ **Optimized Performance** — Efficient memory management and faster processing for large image sets (optional Cython/C++ convolution core).
-- 🕰 **Legacy Mode** — Ensures full backward compatibility with MATLAB’s original SHINE toolbox.
-- 🔢 **High-Precision Arithmetic** — Computations in floating-point precision rather than 8-bit integer space, minimizing rounding errors in multi-stage processing.
-- 📦 **Object-Oriented Design** — Modular, extensible architecture with a clean Python API.
-- 😀 **User-Friendly CLI** — Guided, prompt-based interface for users who prefer not to write code.
+- **Color Processing** — New modes for color image control with modern color-space standards (Rec.601 / Rec.709 / Rec.2020).
+- **Dithering Support** — Reduces quantization artifacts and enhances output image quality.
+- **Optimized Performance** — Efficient memory management and faster processing for large image sets (optional Cython/C++ convolution core).
+- **Legacy Mode** — Ensures full backward compatibility with MATLAB’s original SHINE toolbox.
+- **High-Precision Arithmetic** — Computations in floating-point precision rather than 8-bit integer space, minimizing rounding errors in multi-stage processing.
+- **Object-Oriented Design** — Modular, extensible architecture with a clean Python API.
+- **User-Friendly CLI** — Guided, prompt-based interface for users who prefer not to write code.
For detailed technical documentation (algorithms, numerical choices, and MATLAB vs Python behavior), see
[`documentation/documentation.md`](documentation/documentation.md).
---
-## 🚀 Quick Start
+## Quick Start
### Installation
@@ -68,7 +68,7 @@ print("shinier version:", getattr(shinier, "__version__", "unknown"))
```
-### 😀 **User-friendly Interface**
+### User-friendly Interface
Call the following bash command to quickly start using the interactive CLI.
```bash
shinier --show_results --image_index=1
@@ -77,7 +77,7 @@ shinier --show_results --image_index=1
-### 🧩 Example in Python
+### Example in Python
Run the following python code to make sure the package is running properly.
```python
from shinier import Options, ImageDataset, ImageProcessor, utils
@@ -110,7 +110,7 @@ Below is an example of results obtained using mode 5 with joint histogram equali