From aa7b75c8866cb8a466e607ea6f102035cbcfa4c1 Mon Sep 17 00:00:00 2001 From: lbgee Date: Fri, 17 Jul 2026 00:20:56 -0700 Subject: [PATCH] Add detailed step reference docs, grouped by category Hand-written reference for all 28 steps plus combine_runs. One index page (step contract, array-shape vocabulary, master table) and four category pages: loading & filtering, detector & spatial, binning, spectra & reductions. Each entry lists what the step reads, what it writes, and every tunable parameter with its default. Wire a Step reference nav section into mkdocs.yml after the YAML guide; keep source_analysis.md as the source-linked API. --- docs/steps/binning.md | 127 +++++++++++++++++++++++++++ docs/steps/detector.md | 145 ++++++++++++++++++++++++++++++ docs/steps/index.md | 93 ++++++++++++++++++++ docs/steps/loading_filtering.md | 150 ++++++++++++++++++++++++++++++++ docs/steps/spectra.md | 89 +++++++++++++++++++ mkdocs.yml | 6 ++ 6 files changed, 610 insertions(+) create mode 100644 docs/steps/binning.md create mode 100644 docs/steps/detector.md create mode 100644 docs/steps/index.md create mode 100644 docs/steps/loading_filtering.md create mode 100644 docs/steps/spectra.md diff --git a/docs/steps/binning.md b/docs/steps/binning.md new file mode 100644 index 0000000..bcb6692 --- /dev/null +++ b/docs/steps/binning.md @@ -0,0 +1,127 @@ +# Axes, bin indices & binned reductions + +Steps that build delay/energy axes, assign each shot to a bin, and collapse +per-shot data into a binned spectrum. Axis and index steps run first; the +`reduce_detector_*` steps consume the indices they produce. See the +[step reference overview](index.md) for the shape vocabulary. + +## Building axes and bin indices + +### `time_binning` +Compute per-shot delays from laser timing keys and lay down delay bins. + +- **Reads:** `lxt_key`, `fast_delay_key`, `tt_correction_key` — each `(shots,)`. + Uses whichever exist. +- **Writes:** `delays` `(shots,)`, `time_bins`, `time_bins_centered`, + `timing_bin_indices` `(shots,)`. +- **Parameters:** + +| name | default | description | +|------|---------|-------------| +| `bins` | required | `"auto"`, an explicit list of centers, or `[min, max, num_points]` | +| `lxt_key` | `"lxt_ttc"` | long-delay stage key; `null` to skip | +| `fast_delay_key` | `"encoder"` | fast-stage key | +| `tt_correction_key` | `"time_tool_correction"` | time-tool jitter correction key | +| `resolution` | `50e-15` | bin width in seconds for `bins: auto` | + +### `make_ccm_axis` +Build incident-energy (CCM) bin edges and centers for an XAS scan. + +- **Reads:** `ccm_key` `(shots,)` when `energies: auto`. +- **Writes:** `ccm_bins` — `n+1` edges; `ccm_energies` — `n` centers. +- **Parameters:** + +| name | default | description | +|------|---------|-------------| +| `energies` | `"auto"` | `"auto"`, explicit list, or `[min, max, num_points]` | +| `ccm_key` | `"ccm"` | per-shot incident-energy key | +| `resolution` | `0.001` | bin width (keV) for `energies: auto` | + +### `ccm_binning` +Assign each shot to a CCM energy bin. + +- **Reads:** `ccm_key` `(shots,)`, `ccm_bins_key` (edges). +- **Writes:** `ccm_bin_indices` `(shots,)`. +- **Parameters:** + +| name | default | description | +|------|---------|-------------| +| `ccm_key` | `"ccm"` | per-shot incident-energy key | +| `ccm_bins_key` | `"ccm_bins"` | edges produced by `make_ccm_axis` | + +### `bin_uniques` +Bin an arbitrary scan variable by its unique values (one bin per value). + +- **Reads:** scan variable `on` `(shots,)`. +- **Writes:** `scanvar_indices` `(shots,)`, `scanvar_bins` (unique values). +- **Parameters:** + +| name | default | description | +|------|---------|-------------| +| `on` | required | per-shot scan variable to bin | + +### `make_energy_axis` +Convert pixel index to emission energy from von Hamos crystal geometry. + +- **Reads:** `detector_key` (to read pixel count) or `n_pixels` directly. +- **Writes:** `_energy` `(pixels,)`. +- **Parameters:** + +| name | default | description | +|------|---------|-------------| +| `crystal_detector_distance` | required | crystal-to-detector distance A (mm) | +| `crystal_radius` | required | crystal bend radius R (mm) | +| `d_spacing` | required | crystal d-spacing (angstrom) | +| `detector_key` | `None` | key to read pixel count from | +| `n_pixels` | `None` | pixel count; overrides `detector_key` | +| `mm_per_pixel` | `0.05` | pixel pitch (mm) | +| `name` | `"xes"` | output prefix, so the axis is `_energy` | + +## Binned reductions + +Each reduction sums per-shot data into its bins. Pass `average: True` to divide +by the per-bin count; the raw count is always written as `_..._bincount` for +downstream normalization or cross-run combination. + +### `reduce_detector_temporal` +Bin a per-shot spectrum along delay. + +- **Reads:** detector `on` — 1D `(shots,)` or 2D `(shots, pixels)`; plus + `timing_bin_indices` and `time_bins`. +- **Writes:** `_time_binned` `(n_time_bins[, pixels])`, `_bincount`. +- **Parameters:** + +| name | default | description | +|------|---------|-------------| +| `on` | required | detector key | +| `timing_bin_key` | `"timing_bin_indices"` | bin-index attribute | +| `average` | `False` | divide each bin by its shot count | + +### `reduce_detector_ccm` +Bin a per-shot spectrum along incident energy. + +- **Reads:** detector `on` — 1D/2D/3D; plus `ccm_bin_indices`. +- **Writes:** `_energy_binned` `(n_energy[, ...])`, `_energy_bincount`. +- **Parameters:** + +| name | default | description | +|------|---------|-------------| +| `on` | required | detector key | +| `ccm_bin_key` | `"ccm_bin_indices"` | bin-index attribute | +| `average` | `False` | divide each bin by its shot count | + +### `reduce_detector_ccm_temporal` +Bin a per-shot spectrum along both delay and incident energy (2D map). + +- **Reads:** detector `on` — 1D/2D; plus `timing_bin_indices` and + `ccm_bin_indices`. +- **Writes:** `_time_energy_binned` `(n_time, n_energy[, pixels])`, + `_time_energy_bincount`. +- **Parameters:** + +| name | default | description | +|------|---------|-------------| +| `on` | required | detector key | +| `timing_bin_key` | `"timing_bin_indices"` | delay bin-index attribute | +| `ccm_bin_key` | `"ccm_bin_indices"` | energy bin-index attribute | +| `average` | `False` | divide each bin by its shot count | diff --git a/docs/steps/detector.md b/docs/steps/detector.md new file mode 100644 index 0000000..fa73a82 --- /dev/null +++ b/docs/steps/detector.md @@ -0,0 +1,145 @@ +# Detector correction, geometry & spatial reduction + +Steps that clean up detector frames, straighten a tilted dispersion axis, and +collapse the spatial axes into a spectrum. See the +[step reference overview](index.md) for the shape vocabulary and the read/write +model. + +## Correction + +### `common_mode_correction` +Subtract a per-row, per-column, or per-bank baseline estimated from a +signal-free band. Shape preserved. + +- **Reads:** detector `on` — 3D `(shots, rows, cols)` or 2D `(rows, cols)`. +- **Writes:** overwrites `on`; same shape. +- **Parameters:** + +| name | default | description | +|------|---------|-------------| +| `on` | required | detector key | +| `axis` | `"row"` | `row` (per-row offset across columns), `column`, or `bank` | +| `method` | `"median"` | `median` (robust) or `mean` | +| `reference` | full extent | `[start, end]` of the dark band, indexed on the axis orthogonal to `axis` | +| `bank_size` | `128` | column width of an ePix100 bank; only used when `axis: bank` | + +The `reference` range is a column range for `axis: row` and a row range for +`axis: column` or `axis: bank`. + +```yaml +- step: common_mode_correction + on: epix + axis: row + reference: [0, 40] # dark columns +``` + +### `patch_pixels` +Repair bad pixels or columns, either from an explicit list or auto-detected. + +- **Reads:** detector `on` — 1D, 2D, or 3D. +- **Writes:** overwrites `on`; same shape. +- **Parameters:** + +| name | default | description | +|------|---------|-------------| +| `on` | required | detector key | +| `pixels` | `None` | explicit list of indices/columns to patch | +| `auto_detect` | `False` | find bad columns from the data instead of a list | +| `threshold` | `5.0` | auto-detect: deviation cut for flagging a column | +| `nsigma` | `5.0` | auto-detect: sigma multiplier for the flag | +| `max_gap_width` | `4` | widest run of bad pixels to bridge | +| `smooth_window` | `31` | window for the smoothed reference profile | +| `mode` | `"polynomial"` | `polynomial`, `interpolate`, or `zero` | +| `axis` | last | axis to patch along | +| `patch_range` | `4` | half-width of the neighborhood sampled around a bad pixel | +| `poly_range` | `6` | half-width of the fit window | +| `deg` | `1` | polynomial degree for `mode: polynomial` | + +## Geometry + +### `find_rotation_angle` +Auto-detect the tilt of a dispersed signal so `rotate_detector` can straighten +it. + +- **Reads:** detector `on` — 3D `(shots, rows, cols)` or 2D. +- **Writes:** `_angle` (override with `angle_key`) — float degrees. +- **Parameters:** + +| name | default | description | +|------|---------|-------------| +| `on` | required | detector key | +| `low_threshold` | `30` | lower ADU bound for edge detection | +| `high_threshold` | `100` | upper ADU bound for edge detection | +| `angle_key` | `_angle` | attribute to store the detected angle | + +### `rotate_detector` +Rotate detector frames by a fixed angle or by a previously detected one. + +- **Reads:** detector `on` — 3D or 2D. +- **Writes:** overwrites `on`. +- **Parameters:** + +| name | default | description | +|------|---------|-------------| +| `on` | required | detector key | +| `angle` | `0` | rotation in degrees | +| `angle_key` | `None` | read the angle from this attribute; takes precedence over `angle` | +| `axes` | `[1, 2]` 3D / `[0, 1]` 2D | plane to rotate in | +| `reshape` | `False` | grow the output to fit the rotated frame vs keep shape | + +```yaml +- step: find_rotation_angle + on: epix + angle_key: epix_angle +- step: rotate_detector + on: epix + angle_key: epix_angle +``` + +## Spatial reduction + +### `apply_roi` +Crop to one or more regions of interest, keeping the spatial dimension. + +- **Reads:** detector `on` — 2D or 3D. +- **Writes:** `_ROI_n` (one per ROI, or combined) — spatial axes retained. +- **Parameters:** + +| name | default | description | +|------|---------|-------------| +| `on` | required | detector key | +| `rois` | required | list of `[row0, row1, col0, col1]` crops | +| `combine_rois` | `True` | merge ROIs into one array vs keep separate `_ROI_n` | + +### `reduce_detector_spatial` +Crop to ROIs and collapse one spatial axis into a per-shot dispersion trace. + +- **Reads:** detector `on` — 3D `(shots, rows, cols)` or 2D. +- **Writes:** `_ROI_n` — reduced along the chosen axis. +- **Parameters:** + +| name | default | description | +|------|---------|-------------| +| `on` | required | detector key | +| `rois` | required | list of `[row0, row1, col0, col1]` crops | +| `combine_rois` | `True` | merge ROIs vs keep separate | +| `reduction` | `"sum"` | `sum` or `mean` over the reduced axis | +| `axis` | `1` (3D) / `-1` | spatial axis to collapse | +| `purge` | `True` | drop the source key after reducing to free memory | + +ROI rows are given in absolute detector coordinates; the step translates them +into the loaded crop using the run's `_row_offset`, so the same YAML works +whether the detector was cropped at load or not. + +### `reduce_detector_shots` +Collapse the shot axis into a single averaged or summed frame. + +- **Reads:** detector `on` — shot axis is axis 0. +- **Writes:** `_reduced` — shot axis removed. +- **Parameters:** + +| name | default | description | +|------|---------|-------------| +| `on` | required | detector key | +| `reduction` | `"sum"` | `sum` or `mean` over shots | +| `purge` | `True` | drop the source key after reducing | diff --git a/docs/steps/index.md b/docs/steps/index.md new file mode 100644 index 0000000..4ae1d2e --- /dev/null +++ b/docs/steps/index.md @@ -0,0 +1,93 @@ +# Step reference + +Every registered pipeline step, grouped by what it does, with the arrays it +reads, the arrays it writes, and the parameters you set in YAML. + +For the auto-generated, source-linked docstrings see +[Analysis steps (API)](../source_analysis.md). For how steps fit into a full +pipeline file, see the [YAML pipeline guide](../YAML_PIPELINE_GUIDE.md). + +## The step contract + +A step is a stateless function `step(run, **kwargs) -> None`. The `run` object +is a shared blackboard: a step reads attributes off it, does its work, and +writes results back as new attributes. The next step picks those up by name. +Nothing is returned; everything flows through `run`. + +So in this reference: + +- **Reads** = the run attributes (and their shapes) a step expects to already exist. +- **Writes** = the run attributes it sets. +- **Parameters** = the keys you pass in the YAML `step:` block. `on` is almost + always the attribute a step operates on. + +A step that can't find its input (`on` missing, or the attribute is `None`) +returns quietly and logs a status message rather than raising. This lets the +same pipeline run on a parent that has no detector loaded (batched path) and on +each batch that does. + +## Array shapes + +The docs use a consistent vocabulary for array shapes: + +| Shape | Meaning | +|-------|---------| +| `(shots, rows, cols)` | 3D detector, one frame per shot | +| `(shots, pixels)` | 2D detector after one spatial axis is reduced | +| `(shots,)` | per-shot scalar or 1D key (IPM, delay, CCM energy) | +| `(shots,)` bool | shot mask (`xray`, `laser`, `simultaneous`) | +| `(n_time_bins, pixels)` or `(n_time_bins,)` | time-binned spectrum | +| `(n_energy, pixels)` or `(n_energy,)` | energy (CCM) binned spectrum | +| `(n_time, n_energy[, pixels])` | 2D time+energy binned | + +Detector steps generally accept 2D or 3D and act on the shot axis (axis 0) or +the last (dispersion) axis; each entry says which. + +## Naming conventions + +Steps build output names from the input key plus a suffix describing the +operation, so a chain reads left to right: + +``` +epix (loaded detector, 3D) +epix_ROI_1 (reduce_detector_spatial) +epix_ROI_1_simultaneous_laser (union_shots) +epix_ROI_1_simultaneous_laser_time_binned (reduce_detector_temporal) +epix_ROI_1_simultaneous_laser_time_binned_normalized (normalize_xes) +``` + +Each step's `on` is the previous step's output name. + +## All steps + +| Step | Group | Reads | Writes | +|------|-------|-------|--------| +| [`load_run_keys`](loading_filtering.md#load_run_keys) | Loading | HDF5 | named per-shot keys | +| [`load_detector`](loading_filtering.md#load_detector) | Loading | HDF5 | 3D detector | +| [`get_run_shot_properties`](loading_filtering.md#get_run_shot_properties) | Loading | lightStatus | `xray`, `laser`, `simultaneous` | +| [`droplet_reconstruction`](loading_filtering.md#droplet_reconstruction) | Loading | sparse HDF5 | `new_key` 3D stack | +| [`filter_shots`](loading_filtering.md#filter_shots) | Filtering | mask + key | overwrites mask | +| [`filter_detector_adu`](loading_filtering.md#filter_detector_adu) | Filtering | detector | overwrites `on` | +| [`filter_detector_variance`](loading_filtering.md#filter_detector_variance) | Filtering | 3D detector | overwrites `on` + `_variance_mask` | +| [`hitfinding`](loading_filtering.md#hitfinding) | Filtering | 3D detector | overwrites `on` (fewer shots) | +| [`union_shots`](loading_filtering.md#union_shots) | Filtering | `on` + masks | `new_key` | +| [`separate_shots`](loading_filtering.md#separate_shots) | Filtering | `on` + 2 masks | `new_key` | +| [`common_mode_correction`](detector.md#common_mode_correction) | Detector | 3D/2D detector | overwrites `on` | +| [`patch_pixels`](detector.md#patch_pixels) | Detector | detector | overwrites `on` | +| [`find_rotation_angle`](detector.md#find_rotation_angle) | Detector | 3D/2D detector | `_angle` | +| [`rotate_detector`](detector.md#rotate_detector) | Detector | 3D/2D detector | overwrites `on` | +| [`apply_roi`](detector.md#apply_roi) | Spatial | 2D/3D detector | `_ROI_n` (keeps spatial) | +| [`reduce_detector_spatial`](detector.md#reduce_detector_spatial) | Spatial | 3D/2D detector | `_ROI_n` (reduced) | +| [`reduce_detector_shots`](detector.md#reduce_detector_shots) | Spatial | detector | `_reduced` | +| [`time_binning`](binning.md#time_binning) | Binning | timing keys | `time_bins`, `timing_bin_indices` | +| [`make_ccm_axis`](binning.md#make_ccm_axis) | Binning | `ccm` | `ccm_bins`, `ccm_energies` | +| [`ccm_binning`](binning.md#ccm_binning) | Binning | `ccm`, `ccm_bins` | `ccm_bin_indices` | +| [`bin_uniques`](binning.md#bin_uniques) | Binning | scan var | `scanvar_indices`, `scanvar_bins` | +| [`make_energy_axis`](binning.md#make_energy_axis) | Binning | detector shape | `_energy` | +| [`reduce_detector_temporal`](binning.md#reduce_detector_temporal) | Binning | 1D/2D + time indices | `_time_binned` | +| [`reduce_detector_ccm`](binning.md#reduce_detector_ccm) | Binning | 1D/2D/3D + ccm indices | `_energy_binned` | +| [`reduce_detector_ccm_temporal`](binning.md#reduce_detector_ccm_temporal) | Binning | 1D/2D + both indices | `_time_energy_binned` | +| [`normalize_xes`](spectra.md#normalize_xes) | Spectra | 1D/2D spectrum | `_normalized` | +| [`subtract_polynomial_background`](spectra.md#subtract_polynomial_background) | Spectra | 1D/2D spectrum | `_bkgsub` | +| [`purge_keys`](spectra.md#purge_keys) | Utility | nothing | sets keys to `None` | +| [`combine_runs`](spectra.md#combine_runs) | Reduction | per-run binned data | results dict | diff --git a/docs/steps/loading_filtering.md b/docs/steps/loading_filtering.md new file mode 100644 index 0000000..bdbff27 --- /dev/null +++ b/docs/steps/loading_filtering.md @@ -0,0 +1,150 @@ +# Loading & shot selection + +Steps that pull data off disk and steps that decide which shots survive. See +the [step reference overview](index.md) for the shape vocabulary and the +read/write model. + +## Loading + +### `load_run_keys` +Load scalar and 1D keys from the smalldata HDF5 into named run attributes. + +- **Reads:** nothing on the run; pulls the listed HDF5 paths from `run.run_file`. +- **Writes:** one `(shots,)` attribute per key, named by `friendly_names`. +- **Parameters:** + +| name | default | description | +|------|---------|-------------| +| `keys` | `[]` | list of HDF5 dataset paths | +| `friendly_names` | `[]` | attribute names to store them under (parallel to `keys`) | + +### `load_detector` +Load a 3D detector stack (one image per shot) with optional crop and transpose. + +- **Reads:** nothing on the run; reads the HDF5 paths (delayed). +- **Writes:** a `(shots, rows, cols)` attribute per detector, named by `friendly_names`. +- **Parameters:** + +| name | default | description | +|------|---------|-------------| +| `keys` | `[]` | HDF5 detector paths | +| `friendly_names` | `[]` | attribute names | +| `transpose` | `False` | swap rows/cols on load | +| `rois` | `None` | list of `[start, end]` ranges to crop at load | +| `combine_rois` | `True` | merge ROIs into one array vs keep separate | + +### `get_run_shot_properties` +Load the per-shot x-ray / laser status masks from `lightStatus`. + +- **Reads:** `lightStatus` (via the run). +- **Writes:** `xray`, `laser`, `simultaneous` — each `(shots,)` bool. +- **Parameters:** none. + +### `droplet_reconstruction` +Rebuild dense per-shot images from `droplet2photon` sparse photon positions, +reading directly from the source HDF5. Output plugs into every downstream +detector step. + +- **Reads:** sparse photon arrays from `run.run_file`. In batch mode the batch + manager injects `abs_start_index` / `abs_end_index`; otherwise the range comes + from `run.start_index` / `run.end_index`. +- **Writes:** `new_key` — `(n_shots, rows, cols)`. +- **Parameters:** + +| name | default | description | +|------|---------|-------------| +| `det` | required | detector group in the HDF5, e.g. `epix100_0` | +| `new_key` | required | attribute to store the reconstructed stack | +| `roi` | `None` | `[row0, row1, col0, col1]` crop; omit for full panel | +| `panel_shape` | `[704, 768]` | full panel `[rows, cols]` | + +```yaml +- step: droplet_reconstruction + det: epix100_0 + new_key: epix_spec + roi: [270, 330, 400, 700] # -> (60, 300) output +``` + +## Filtering & shot selection + +### `filter_shots` +Tighten a shot mask by thresholding on another per-shot key. NaNs in the +filter key are dropped. + +- **Reads:** the mask named by `on` `(shots,)`, and `filter_key` `(shots,)`. +- **Writes:** overwrites the mask `on` (fewer `True` entries). +- **Parameters:** + +| name | default | description | +|------|---------|-------------| +| `on` | required | shot mask key, e.g. `xray`, `simultaneous` | +| `filter_key` | `"ipm"` | per-shot key to threshold on | +| `threshold` | `1e4` | scalar (keep `> threshold`) or `[min, max]` (keep inside) | + +### `filter_detector_adu` +Zero detector pixels outside an ADU window. Shape unchanged. + +- **Reads:** detector `on` (any dimensionality). +- **Writes:** overwrites `on`; sub-threshold pixels set to 0. +- **Parameters:** + +| name | default | description | +|------|---------|-------------| +| `on` | required | detector key | +| `adu_threshold` | `3.0` | scalar (keep `> t`) or `[min, max]` (keep inside) | + +### `filter_detector_variance` +Zero pixels whose value barely changes across shots (dead/hot/constant pixels). +Data-driven alternative to a hand-tuned ADU cut, using sklearn +`VarianceThreshold`. + +- **Reads:** 3D detector `on` `(shots, rows, cols)` (flattened to shots × features). +- **Writes:** overwrites `on`; also `_variance_mask` `(rows, cols)` bool of retained pixels. +- **Parameters:** + +| name | default | description | +|------|---------|-------------| +| `on` | required | detector key | +| `variance_threshold` | `0.0` | pixels with variance `<=` this are zeroed; `0.0` removes only constant pixels | + +### `hitfinding` +Keep only shots whose total detector signal clears a threshold. + +- **Reads:** 3D detector `on` `(shots, rows, cols)`. +- **Writes:** overwrites `on` with the surviving shots (first axis shrinks). +- **Parameters:** + +| name | default | description | +|------|---------|-------------| +| `on` | required | detector key | +| `min_sum` | `None` | absolute per-shot ADU floor; takes precedence when set | +| `cutoff_multiplier` | `1.0` | relative threshold `median - k*std` when `min_sum` is unset | + +Use `min_sum: 1.0` to drop shots that are all-zero after ADU filtering. The +relative mode breaks down when most shots are dark (median ≈ 0). + +### `union_shots` +Keep shots where ALL listed masks are true (logical AND). + +- **Reads:** `on` (shot axis 0) and each mask in `filter_keys` `(shots,)`. +- **Writes:** `new_key` (default `__`), subset along shots. +- **Parameters:** + +| name | default | description | +|------|---------|-------------| +| `on` | required | data/detector key to filter | +| `filter_keys` | `[]` | mask names to AND together | +| `new_key` | auto | output name; defaults to `_` | + +### `separate_shots` +Keep shots matching the first mask but NOT the second (A and not B). + +- **Reads:** `on` and the two masks in `filter_keys`. +- **Writes:** `new_key` (default `__not_`). +- **Parameters:** + +| name | default | description | +|------|---------|-------------| +| `on` | required | data/detector key | +| `filter_keys` | `[]` | `[include_mask, exclude_mask]` | +| `new_key` | auto | output name; defaults to `__not_` | diff --git a/docs/steps/spectra.md b/docs/steps/spectra.md new file mode 100644 index 0000000..db7a9af --- /dev/null +++ b/docs/steps/spectra.md @@ -0,0 +1,89 @@ +# Spectra, utility & cross-run reductions + +Steps that finish a binned spectrum (normalize, background-subtract), a utility +step to drop keys, and the one reduction that runs across all completed runs. +See the [step reference overview](index.md) for the shape vocabulary. + +## Spectra + +### `normalize_xes` +Area-normalize each spectrum so it sums to 1 over a pixel range. + +- **Reads:** spectrum `on` — 1D `(pixels,)` or 2D `(bins, pixels)`. +- **Writes:** `_normalized`; also `_normalized_std` when a matching + `_std` key exists. +- **Parameters:** + +| name | default | description | +|------|---------|-------------| +| `on` | required | spectrum key (usually `_time_binned`) | +| `pixel_range` | full range | `[start, end]` pixels the sum is taken over | + +For 2D input each row (bin) is divided by its own sum; zero-sum rows are left +unchanged. If `` carries a companion `_std` array, it is scaled by the same +factor and written as `_normalized_std`. + +### `subtract_polynomial_background` +Fit a polynomial baseline along the spatial axis and subtract it. +Non-destructive. + +- **Reads:** spectrum `on` — 1D `(pixels,)` or 2D `(bins, pixels)`. +- **Writes:** `_bkgsub`; same shape. +- **Parameters:** + +| name | default | description | +|------|---------|-------------| +| `on` | required | spectrum key | +| `axis` | last | spatial axis to fit along | +| `order` | `2` | polynomial degree | +| `background` | `None` | list of `[start, end]` signal-free ranges to fit | +| `peak_mask` | `None` | range(s) to EXCLUDE; single `[start, end]` or a list of ranges | + +Give either `background` (the regions to fit) or `peak_mask` (the regions to +skip). `peak_mask` takes a list of ranges so multiple dispersed lines, e.g. +Kalpha and Kbeta on one detector, are masked together. `background` wins if +both are set. + +```yaml +- step: subtract_polynomial_background + on: epix_ROI_1_time_binned + order: 2 + peak_mask: [[120, 180], [300, 360]] # two emission lines +``` + +## Utility + +### `purge_keys` +Set the listed run attributes to `None` to free memory mid-pipeline. + +- **Reads:** nothing. +- **Writes:** sets each named attribute to `None`. +- **Parameters:** + +| name | default | description | +|------|---------|-------------| +| `keys` | `[]` | list of attribute names to drop | + +## Cross-run reduction + +### `combine_runs` +A reduction, not a step: runs once after all per-run pipelines finish, summing +laser-on and laser-off binned data across runs and returning a results dict. + +- **Signature:** `reduction(runs) -> dict` (operates on the list of completed + runs, not a single `run`). +- **Reads (per run):** ``, + ``, and their `_bincount` arrays. +- **Returns:** a dict with `laser_on_summed`, `laser_off_summed`, + `laser_on_count`, `laser_off_count`, and, when all four are present, + `laser_on_average`, `laser_off_average`, and `difference` + `((on - off) / off)`. +- **Parameters:** + +| name | default | description | +|------|---------|-------------| +| `detector_key` | `"epix_ROI_1"` | base detector key | +| `laser_on_suffix` | `"_simultaneous_laser_time_binned"` | laser-on data suffix | +| `laser_off_suffix` | `"_xray_not_laser_time_binned"` | laser-off data suffix | + +Results land in `pipe.results` under the reduction name. diff --git a/mkdocs.yml b/mkdocs.yml index 57c35a5..f58a1ea 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -12,6 +12,12 @@ nav: - Home: index.md - Overview: Overview.md - YAML pipeline guide: YAML_PIPELINE_GUIDE.md + - Step reference: + - Overview: steps/index.md + - Loading & filtering: steps/loading_filtering.md + - Detector & spatial: steps/detector.md + - Binning: steps/binning.md + - Spectra & reductions: steps/spectra.md - Examples: - XAS: Getting_started_XAS.md - XES: Getting_started_XES.md