Skip to content

Implement Ensemble-Based Multi-Track Finder - #26

Draft
dhshin04 wants to merge 2 commits into
devfrom
feat/ensemble-fixed-n-track-finder
Draft

Implement Ensemble-Based Multi-Track Finder#26
dhshin04 wants to merge 2 commits into
devfrom
feat/ensemble-fixed-n-track-finder

Conversation

@dhshin04

@dhshin04 dhshin04 commented Jun 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

Implements a fixed-N ensemble approach to multi-track dimuon pair finding, decomposing the problem into two independently trainable components:

  • Track counter (models/TrackCounter.py): classifies how many ground-truth pairs (0..N) are in each event, using the denoiser U-Net++ backbone with a GlobalAveragePooling → Dense head and sparse-CE loss
  • Fixed-N finders (models/MultiTrackFinder.py): unified code path for all N (1–3); output always (B, N, 2, 62, 201); trained with the new min_perm_loss — a brute-force permutation-invariant loss (N! ≤ 6) that finds the minimum-cost assignment of predicted pair slots to ground-truth slots

Key changes

File Change
data/multi_track/gen_fixed_pairs.py New: fixed-pair data generator (fork of gen_training_random.py)
models/losses.py Add min_perm_loss(n_pairs) — permutation-invariant loss; exact match to custom_loss at N=1
models/data_loader.py Add load_data_counter() — returns (X, counts) for counter training
models/MultiTrackFinder.py Rewrite: unified N path, --n_pairs replaces --max_pairs / presence loss args
models/TrackCounter.py New: classification model with curriculum learning and sklearn confusion report
eval_multi_track.py Rewrite: permutation-matched metrics per pair, no chi-squared, --n_pairs arg
scripts/preprocess_fixed.slurm New: generates fixed-N and random training data for all N
scripts/train_counter.slurm New: counter training job (1× A100)
scripts/train_finder.slurm New: parameterized finder training + eval (4× A100, N_PAIRS env var)

Design decisions

  • N=1 exact equivalence: min_perm_loss(1) is mathematically identical to the original custom_loss (same CE + overlap penalty scalar), so N=1 can be retrained through the new code without regressions
  • Orchestrator deferred: counter and finders are trained and evaluated independently; the routing orchestrator is out of scope for this PR
  • Brute-force permutations: N ≤ 3 gives at most 3! = 6 permutations, making Hungarian matching unnecessary

Test plan

  • Run gen_fixed_pairs.py on a small sample; verify HitArray_mup has length N*62 and nPairs == N every event
  • Confirm min_perm_loss(1) value matches custom_loss on identical input (after squeezing the N=1 pair axis)
  • Smoke-train MultiTrackFinder.py --n_pairs 1 and --n_pairs 2 for 1 epoch; confirm output shapes (B,1,2,62,201) and (B,2,2,62,201)
  • Smoke-train TrackCounter.py for 1 epoch; confirm confusion matrix prints at end
  • Run eval_multi_track.py --n_pairs 2 on a checkpoint; confirm per-pair tables print and no chi-squared appears
  • Submit train_finder.slurm with N_PAIRS=1, N_PAIRS=2, N_PAIRS=3 on HPC

dhshin04 added 2 commits June 7, 2026 14:00
Five-component ensemble framework for finding multiple dimuon pairs per
event, replacing the dynamic-occupancy MultiTrackFinder:

- data/multi_track/gen_fixed_pairs.py: fork of gen_training_random.py
  that pins current_pairs=N (fixed) with random extra single-muon noise
  within the remaining budget; HitArray sized to exactly (N,62)

- models/losses.py: add min_perm_loss(n_pairs) — permutation-invariant
  loss that brute-forces N! assignments (N<=3) to find minimum-cost
  pair slot matching; reproduces custom_loss exactly at N=1

- models/data_loader.py: add load_data_counter() returning (X, counts)
  for training the track counter classifier

- models/MultiTrackFinder.py: unified N code path (no N=1 special case);
  output always (B,N,2,62,201); uses min_perm_loss; CLI --n_pairs replaces
  --max_pairs/--lambda_presence/--pos_weight_presence

- models/TrackCounter.py: new classification model predicting number of
  dimuon pairs (0..max_pairs) using denoiser U-Net++ backbone +
  GlobalAveragePooling2D + Dense head with curriculum learning

- eval_multi_track.py: rewrite with brute-force permutation matching
  before metrics, per-pair reporting, no chi-squared; --n_pairs arg

- scripts/preprocess_fixed.slurm: generates fixed-N and random data
- scripts/train_counter.slurm: trains TrackCounter (single A100)
- scripts/train_finder.slurm: trains MultiTrackFinder for given N (4xA100),
  parameterized via N_PAIRS env var, runs eval after training
@dhshin04 dhshin04 changed the title feat: ensemble fixed-N multi-track finder pipeline Implement Ensemble-Based Multi-Track Finder Jun 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant