feat: derive VBPCA hyperparameter recommendations for extreme aspect-ratio regimes - #120
Merged
Merged
Conversation
…ratio regimes (#116) Adds analysis/trade_study/option_a_aspect_ratio.py: uses trade_study.recommend_per_regime()/aggregate_bucketed_config() (jcm-sci/trade-study#123, split in #126) to find good VBPCA hyperparameters for genomics-scale (p>>n) and ecological/survey-scale (n>>p) data -- shapes #116 found the shipped smallp/trans/large buckets were never validated against (p up to 200, p/n up to 2.0 only). search/aggregate are split rather than one combined call: a first 2-bucket (wide vs tall) grouping performed badly for the two most extreme regimes even with n_reps=3 replicate averaging (bulk_rnaseq rank_mae=4.0, ecological rank_mae=5.0), while a second, finer 4-bucket grouping by aspect-ratio extremity -- giving the two extreme regimes their own dedicated bucket instead of averaging them in with milder regimes -- fixed it: regime | wide_tall (2 buckets) | extremity (4 buckets) bulk_rnaseq | rank_mae=4.0 | rank_mae=0.0 microbiome | rank_mae=0.0 | rank_mae=0.0 single_cell | rank_mae=0.0 | rank_mae=0.0 ecological | rank_mae=5.0 | rank_mae=4.0 cultural | rank_mae=1.0 | rank_mae=1.0 Re-running the whole ~2hr NSGA-II search to try a different bucket_fn would have been wasteful, hence the split API in #126: `search` runs once and caches every regime's best config; `aggregate` re-groups from that cache for free with either grouping. Not (yet) wired into defaults.py's shipped _BUCKET_CONFIGS -- that's a separate decision on whether/how to extend recommend_config() itself. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2 tasks
jc-macdonald
added a commit
that referenced
this pull request
Aug 24, 2026
…uckets feat: wire #120's aspect-ratio buckets into recommend_config()
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
analysis/trade_study/option_a_aspect_ratio.py, addressing #116's finding thatrecommend_config's shipped buckets were never validated against genomics-scale (p >> n) or ecological/survey-scale (n >> p) data -- the Option A regime grid only coverspup to 200 andp/nup to 2.0.Approach
Uses
trade_study.recommend_per_regime()/aggregate_bucketed_config()(jcm-sci/trade-study#123, split into two functions in #126) to search VBPCA hyperparameters independently per regime via NSGA-II, then aggregate into named buckets.search/aggregateare separate CLI subcommands rather than one combined call -- re-running the ~2hr NSGA-II search to try a different bucket grouping would have been wasteful, sosearchruns once and caches every regime's best config;aggregatere-groups from that cache for free.Result: 2 buckets isn't enough, 4 is
A first 2-bucket grouping (wide vs tall) performed badly for the two most extreme regimes even with
n_reps=3replicate averaging to guard against single-draw luck. A second, finer 4-bucket grouping by aspect-ratio extremity -- giving the extreme regimes their own dedicated bucket instead of averaging them in with milder ones -- fixed it:bulk_rnaseqgoes from a real failure to perfect recovery;ecologicalimproves substantially (still not perfect, likely because its "best" config was chosen by 3-rep-averaged score during search but validated here on a single draw -- some variance is expected). Both moderate-aspect-ratio regimes and their configs are completely unaffected, and getting worse than 2-bucket wasn't observed anywhere.Not included
Not wired into
defaults.py's shipped_BUCKET_CONFIGS-- whether/how to extendrecommend_config()itself with these buckets is a separate decision (bigger blast radius, changes production behavior for all callers) I'd like to check with the maintainer before doing.Test plan
just cipassesVBPCASimulator/VBPCAScorer(table above)🤖 Generated with Claude Code