#

fineweb-edu

Here is 1 public repository matching this topic...

Kevin-Li-2025 / l20-edu-135m-pretrain

From-scratch 135M Transformer pretraining on 10B FineWeb-Edu tokens using a single NVIDIA L20, with public checkpoint and lm-eval comparisons.

pytorch transformer reproducibility pretraining l20 llm-training lm-eval fineweb-edu

Updated Jun 27, 2026
Python

Improve this page

Add a description, image, and links to the fineweb-edu topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the fineweb-edu topic, visit your repo's landing page and select "manage topics."