Skip to content

feat: derive VBPCA hyperparameter recommendations for extreme aspect-ratio regimes - #120

Merged
jc-macdonald merged 1 commit into
mainfrom
feat/116-aspect-ratio-buckets
Aug 22, 2026
Merged

feat: derive VBPCA hyperparameter recommendations for extreme aspect-ratio regimes#120
jc-macdonald merged 1 commit into
mainfrom
feat/116-aspect-ratio-buckets

Conversation

@jc-macdonald

Copy link
Copy Markdown
Collaborator

Summary

Adds analysis/trade_study/option_a_aspect_ratio.py, addressing #116's finding that recommend_config's shipped buckets were never validated against genomics-scale (p >> n) or ecological/survey-scale (n >> p) data -- the Option A regime grid only covers p up to 200 and p/n up to 2.0.

Approach

Uses trade_study.recommend_per_regime()/aggregate_bucketed_config() (jcm-sci/trade-study#123, split into two functions in #126) to search VBPCA hyperparameters independently per regime via NSGA-II, then aggregate into named buckets. search/aggregate are separate CLI subcommands rather than one combined call -- re-running the ~2hr NSGA-II search to try a different bucket grouping would have been wasteful, so search runs once and caches every regime's best config; aggregate re-groups from that cache for free.

Result: 2 buckets isn't enough, 4 is

A first 2-bucket grouping (wide vs tall) performed badly for the two most extreme regimes even with n_reps=3 replicate averaging to guard against single-draw luck. A second, finer 4-bucket grouping by aspect-ratio extremity -- giving the extreme regimes their own dedicated bucket instead of averaging them in with milder ones -- fixed it:

regime wide_tall (2 buckets) extremity (4 buckets)
bulk_rnaseq (p/n=66.7) rank_mae=4.0, rmse=2.16 rank_mae=0.0, rmse=0.78
microbiome (p/n=6) rank_mae=0.0 rank_mae=0.0
single_cell (p/n=1) rank_mae=0.0 rank_mae=0.0
ecological (n/p=100) rank_mae=5.0 rank_mae=4.0
cultural (n/p=20) rank_mae=1.0 rank_mae=1.0

bulk_rnaseq goes from a real failure to perfect recovery; ecological improves substantially (still not perfect, likely because its "best" config was chosen by 3-rep-averaged score during search but validated here on a single draw -- some variance is expected). Both moderate-aspect-ratio regimes and their configs are completely unaffected, and getting worse than 2-bucket wasn't observed anywhere.

Not included

Not wired into defaults.py's shipped _BUCKET_CONFIGS -- whether/how to extend recommend_config() itself with these buckets is a separate decision (bigger blast radius, changes production behavior for all callers) I'd like to check with the maintainer before doing.

Test plan

  • just ci passes
  • Validated both groupings' configs against all 5 regimes directly via VBPCASimulator/VBPCAScorer (table above)

🤖 Generated with Claude Code

…ratio regimes (#116)

Adds analysis/trade_study/option_a_aspect_ratio.py: uses
trade_study.recommend_per_regime()/aggregate_bucketed_config()
(jcm-sci/trade-study#123, split in #126) to find good VBPCA
hyperparameters for genomics-scale (p>>n) and ecological/survey-scale
(n>>p) data -- shapes #116 found the shipped smallp/trans/large buckets
were never validated against (p up to 200, p/n up to 2.0 only).

search/aggregate are split rather than one combined call: a first
2-bucket (wide vs tall) grouping performed badly for the two most
extreme regimes even with n_reps=3 replicate averaging (bulk_rnaseq
rank_mae=4.0, ecological rank_mae=5.0), while a second, finer 4-bucket
grouping by aspect-ratio extremity -- giving the two extreme regimes
their own dedicated bucket instead of averaging them in with milder
regimes -- fixed it:

  regime        | wide_tall (2 buckets) | extremity (4 buckets)
  bulk_rnaseq    | rank_mae=4.0          | rank_mae=0.0
  microbiome     | rank_mae=0.0          | rank_mae=0.0
  single_cell    | rank_mae=0.0          | rank_mae=0.0
  ecological     | rank_mae=5.0          | rank_mae=4.0
  cultural       | rank_mae=1.0          | rank_mae=1.0

Re-running the whole ~2hr NSGA-II search to try a different bucket_fn
would have been wasteful, hence the split API in #126: `search` runs
once and caches every regime's best config; `aggregate` re-groups from
that cache for free with either grouping.

Not (yet) wired into defaults.py's shipped _BUCKET_CONFIGS -- that's a
separate decision on whether/how to extend recommend_config() itself.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@jc-macdonald
jc-macdonald merged commit cfca170 into main Aug 22, 2026
7 checks passed
@jc-macdonald
jc-macdonald deleted the feat/116-aspect-ratio-buckets branch August 22, 2026 00:09
jc-macdonald added a commit that referenced this pull request Aug 24, 2026
…uckets

feat: wire #120's aspect-ratio buckets into recommend_config()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant