Feat/dsep nichr/less based sampler#10
Open
PaxPerVeritatem wants to merge 68 commits into
Open
Conversation
… into seperate folders
…ts but may not fully work yet.
…begin implementation of LESS step 3
… multiple checkpoints and creates directories for each. Also trying to implement step 3.2
… of step3.1_compute_influence_scores
…amic preprocessing interface for validation gradients computation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR finalizes the native integration of the LESS data selection pipeline into Torchtune. The implementation is fully contained and isolated to avoid any conflicts with
mainexisting repository components.✅ Summary of Changes:
-Added the full LESS data selection pipeline.
-Main entry point: LESS_pipeline.py, which utilizes
PercentageBasedSampler, a subclass ofSelectiveSampler.-All LESS-related functionality is located within the
less/andselection/folders.- Comprehensive documentation is provided for each step of the LESS pipeline, within the respective
step<number>_<name>.py files within thelessfolder.📁 File Structure:
-
less/: Contains all files necessary to run the LESS pipeline.-
selection/:Houses the newly introduced sampler classes, includingPercentageBasedSampler.