Feature/yuxuan cl matching integration

[MadivB]

Summary

I have added the run-cl-matching/ production step that will be auto-installed through the install_everything under adimn. This will run the alpha release version charge-light matching (QL matching) after the existing run-ndlar-flow/ step. It is the same as the one discribed in the technote that can be seen in the technote. I included here three similar workflows with the same architecture, but different geometry, mirroring the existing sim/data × ND/2x2 grid:

Script	Detector / mode
run_cl_matching_ND_sim.sh	ND-LAr, simulation
run_cl_matching_2x2_sim.sh	2x2, simulation
run_cl_matching_2x2_data.sh	2x2, real data

All three wrappers drive the external MadivB/CLMatching_AlphaRelease repo, which install_cl_matching.sh clones into run-cl-matching/CLMatching_AlphaRelease/.

Files

• run-cl-matching/install_cl_matching.sh — clones the CLMatching repo, runs its check_install.py which surfaces the one non-bundled asset (the ~490 MB ND-LAr perceiver, hosted on the CLMatching GitHub Release) with the exact curl command to fetch it.
• run-cl-matching/run_cl_matching_ND_sim.sh — ND simulation.
• run-cl-matching/run_cl_matching_2x2_sim.sh — 2x2 simulation.
• run-cl-matching/run_cl_matching_2x2_data.sh — 2x2 real data (input file taken from ND_PRODUCTION_CLMATCH_DATA_FILE; no upstream chain wiring since real DAQ data isn't produced by the ND_Production pipeline).
• run-cl-matching/.gitignore — excludes the cloned CLMatching_AlphaRelease/ from this repo (mirrors what run-larnd-sim/ and run-ndlar-flow/ do).
• admin/install_everything.sh — chains the new install step inside the detector-specific block.

Conventions followed

• Each run_*.sh sources ../util/reload_in_container.inc.sh + ../util/init.inc.sh, consumes ND_PRODUCTION_* env vars, wraps heavy commands with the run helper for logging/timing, and writes outputs to <OUTDIR_BASE>/run-cl-matching/<OUT_NAME>/{FLOW,PT}/<subDir>/.
• install_cl_matching.sh follows the same error-out-if-installed pattern as install_ndlar_flow.sh / install_larnd_sim.sh.
• Uses the shared NERSC nersc-python (torch pre-installed); no separate venv is built.

Override knobs

• ND_PRODUCTION_CLMATCH_REPO — override the CLMatching install location (defaults to $ND_PRODUCTION_INSTALL_DIR/CLMatching_AlphaRelease).
• ND_PRODUCTION_CLMATCH_BRANCH — override the CLMatching git branch (default main).
• ND_PRODUCTION_CLMATCH_VERSION — 2x2 algorithm version, v1.0 (default, error-matrix) or v2.0 (region-grow + tiebreaker).
• ND_PRODUCTION_CLMATCH_DATA_FILE — 2x2 data input file path (default is the standard dune cfs beam-data reflow file).

Mode A (in-place HDF5) vs Mode B (.pt) for ND

run_cl_matching_ND_sim.sh inspects the FLOW file's calib_prompt_hits and calib_final_hits dtypes. If both reserve t_0 and t_cluster_id (the new dtype format), it copies the file into tmpOutDir, has CLMatching write the two fields directly into the compound HDF5 datasets, then mvs the modified .FLOW.hdf5 to outDir/FLOW/<subDir>/. For older FLOW files without those fields, CLMatching's aggregator falls back to producing a .pt under <CLMatching repo>/output/QLmatchingvAlpha/.

Testing

I did a test run before uploading this to you guys and verified twice end-to-end from an empty folder simulating a fresh NERSC user

Serious things that might worth mentioning and discussion (if this makes you a lot of trouble, I am fine with changing the dafaults)

fireworks4dune's stock slurm/fw_gpu.slurm.sh uses --ntasks-per-node=4 --gpus-per-task=1, giving each fireworks task only 1 GPU. Our wrappers default to 8 workers on 4 GPUs. When a ccl_matching.yaml spec is authored, the accompanying submission line should use --ntasks-per-node=1 --gpus-per-node=4 (or set N_GPUS=1 N_WORKERS_PER_GPU=1 in the base_env if 1-GPU-per-task is preferred — both are supported via env-var overrides).