-
Notifications
You must be signed in to change notification settings - Fork 0
All issues
Issue creation is restricted in this repository
Issues
is:issue state:open
is:issue state:open
Search results
Add a demo GIF / terminal recording to the README
documentationImprovements or additions to documentationImprovements or additions to documentationStatus: Open.#8 In Mirdula18/Granule;Encode throughput: pure-Python encoder is 1.3-3.9x slower than tiktoken
enhancementNew feature or requestNew feature or requestStatus: Open.#7 In Mirdula18/Granule;Corpus scale: ~1.5 MB per language is far short of production
enhancementNew feature or requestNew feature or requestStatus: Open.#6 In Mirdula18/Granule;No streaming: the entire corpus is loaded into memory during training
enhancementNew feature or requestNew feature or requestStatus: Open.#5 In Mirdula18/Granule;Fertility is meaningless for whitespace-free corpora (jsonlines reads ~41)
enhancementNew feature or requestNew feature or requestStatus: Open.#4 In Mirdula18/Granule;Naive/fast train timings use a bounded prefix, not the full corpus
enhancementNew feature or requestNew feature or requestStatus: Open.#3 In Mirdula18/Granule;Pre-tokenization diverges from upstream cl100k on leading-space attachment
bugSomething isn't workingSomething isn't workingStatus: Open.#2 In Mirdula18/Granule;