feat: recommend_bucketed_config() -- discrete regime-bucketed adaptive recommendation - #125
Merged
Merged
Conversation
…e recommendation (#123) RegimeSurrogate interpolates continuously across regime descriptors, but needs reasonably dense training coverage -- RF surrogates extrapolate poorly (effectively flat) outside their training range. There was no equivalent for the sparse case: a handful of named regimes, each far from the others and far from any existing training data, where you just want "find a good config for each one directly, then aggregate into named buckets." Found deriving VBPCA hyperparameter buckets for extreme-aspect-ratio data shapes (yoavram-lab/VBPCApy#116) -- the pattern that actually worked (run_adaptive per regime + median aggregation into buckets) was hand-rolled there; this formalizes it as a reusable trade-study function. recommend_bucketed_config() runs run_adaptive per regime (respecting n_reps, #122), selects each regime's best trial by a primary observable (respecting its Direction), groups regimes via a caller-supplied bucket_fn, and aggregates each bucket's per-regime best configs -- median for continuous/discrete factors, mode for categorical. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #123.
RegimeSurrogateinterpolates continuously across regime descriptors, but needs reasonably dense training coverage -- RF surrogates (its default backend) extrapolate poorly (effectively flat) outside their training range. There was no equivalent for the sparse case: a handful of named regimes, each far from the others and far from any existing training data, where the actual need is "find a good config for each one directly, then aggregate into named buckets."Why this matters
Found deriving VBPCA hyperparameter buckets for extreme-aspect-ratio data shapes (yoavram-lab/VBPCApy#116) -- genomics-scale (
p >> n) and ecological/survey-scale (n >> p) regimes, 2-3 per extreme. The pattern that actually worked was hand-rolled inanalysis/trade_study/option_a_aspect_ratio.py:run_adaptiveper regime, pick the best trial, group into named buckets, aggregate. This formalizes that as a reusable trade-study function.Changes
recommend_bucketed_config(regimes, bucket_fn, world_factory, scorer, factors, observables, *, primary, n_trials=30, n_reps=1, seed=42):run_adaptiveindependently per regime (world_factory(regime)builds a regime-scopedSimulator)primaryobservable, respecting itsDirection(minimize/maximize)bucket_fn(name, regime) -> bucket_nameUses
run_adaptive'sn_reps(#122, already merged) to avoid the single-draw fragility that motivated #122 in the first place.Test plan
just cipasses,regime.pyat 100% coveragebucket_fndoesn't merge them, categorical aggregation picks the mode,Direction.MAXIMIZEis respected, rejects emptyregimes/unknownprimary🤖 Generated with Claude Code