Building a complete LLM from scratch. No libraries. No shortcuts. Just NumPy, math, and first principles.

Build an LLM from scratch with transformers, attention, tokenization, dataset engineering, training, alignment, inference, multimodality, agents, and a runnable PyTorch mini language model.

LLM Which Was Trained On A Consumer GPU

DeepSeek-style MoE + MLA from scratch in PyTorch, with a router-specialization probe (mutual information) and a dense/+MoE/+MLA ablation. Educational, nano-scale, measured.

From-scratch PyTorch: frontier LLM techniques as of 2026-Q1 — the Muon optimizer and Multi-Token Prediction, plus a base BPE tokenizer. Self-contained, self-checking modules.

A large language model built from scratch using PyTorch. Currently, it is in the initial stage of the development process using PyTorch.🔧

面向只有 Python 基础学习者的中文 LLM 从零学习伴侣，配合 rasbt/LLMs-from-scratch，含课程、实验、CLI、个性化计划与测试。

From-scratch PyTorch GPT: the GPT-2 skeleton plus the components of current open LLMs — RMSNorm, RoPE, SwiGLU, GQA, no-bias + tied embeddings. Each isolated and self-checking.

Some attempts, based on MicroGPT.

Building a Large Language Model from the ground up.

GRAFT (Grounded Reasoning-Action File-agent Toolchain): a from-scratch ReAct implementation across Groq's OpenAI-compatible API and the Anthropic Messages API, with persistent memory, sandboxed calculation, and zero-hallucination answering.

An ~8M parameter LLM that talks like a tiny robot vacuum.

A from-scratch implementation of a Large Language Model in Python, built while following freeCodeCamp's Create a Large Language Model from Scratch with Python – Tutorial.