📚 Paper Reading

A curated collection of research papers on AI systems, compilers, architecture, and systems software.

---

Table of Contents

• Deep Learning Compiler
• LLM Inference
• LLM Training
• Deep Learning
• LLM Evaluation & Safety
• LLM for Kernel Optimization
• Agent Systems
• GPU Microarchitecture
• Math Foundations
• Compiler
• Operating Systems
• Hypervisor & Virtualization
• RISC-V

Legend: ✅ = Read  |  ⬜ = To Read  |  📝 = Note Available

Learning roadmap: LLM Training, Inference & Algorithms — Recommended Reading Path

---

🔧 Deep Learning Compiler

Status	Paper	Venue	Links
✅	The Deep Learning Compiler: A Comprehensive Survey	—	Paper / Note
✅	MLIR: Scaling Compiler Infrastructure for Domain Specific Computation	CGO'21	Paper / Note
✅	TIRAMISU: A Polyhedral Compiler for Expressing Fast and Portable Code	CGO'19	Paper / Note
✅	Rammer: Enabling Holistic Deep Learning Compiler Optimizations with rTasks	OSDI'20	Paper / Note
✅	ROLLER: Fast and Efficient Tensor Compilation for Deep Learning	OSDI'22	Paper / Note
✅	BOLT: Bridging The Gap Between Auto-Tuners and Hardware-Native Performance	MLSys'22	Paper / Note
✅	AStitch: Enabling a New Multi-dimensional Optimization Space for Memory-Intensive ML Training and Inference on Modern SIMT Architectures	ASPLOS'22	Paper / Note
✅	AMOS: Enabling Automatic Mapping for Tensor Computations On Spatial Accelerators with Hardware Abstraction	ISCA'22	Paper / Note
✅	Welder: Scheduling Deep Learning Memory Access via Tile-graph	OSDI'23	Paper / Note
✅	Effectively Scheduling Computational Graphs of Deep Neural Networks toward Their Domain-Specific Accelerators	OSDI'23	Paper
✅	Cocktailer: Analyzing and Optimizing Dynamic Control Flow in Deep Learning	OSDI'23	Paper / Note
✅	Chimera: An Analytical Optimizing Framework for Effective Compute-intensive Operators Fusion	HPCA'23	Paper / Note
✅	Graphene: An IR for Optimized Tensor Computations on GPUs	ASPLOS'23	Paper / Note
✅	Uncovering Nested Data Parallelism and Data Reuse in DNN Computation with FractalTensor	SOSP'24	Paper
✅	ThunderKittens: Simple, Fast, and Adorable AI Kernels	—	Paper
✅	Mirage: A Multi-Level Superoptimizer for Tensor Programs	OSDI'25	Paper
✅	PipeThreader: Software-Defined Pipelining for Efficient DNN Execution	OSDI'25	Paper
✅	TileLang: A Composable Tiled Programming Model for AI Systems	—	Paper
✅	Tawa: Automatic Warp Specialization for Modern GPUs with Asynchronous References	arXiv'25	Paper
✅	KPerfIR: Towards an Open and Compiler-centric Ecosystem for GPU Kernel Performance Tooling on Modern AI Workloads	OSDI'25	Paper
⬜	Optimal Software Pipelining and Warp Specialization for Tensor Core GPUs	OSDI'26	Paper
⬜	GraCE: Unlocking CUDA Graphs with Compiler Support for ML Workloads	OSDI'26	Paper

🚀 LLM Inference

Decoding Algorithms

Status	Paper	Why It Matters	Links
⬜	The Curious Case of Neural Text Degeneration	系统比较 greedy、beam、top-k 与 nucleus sampling，理解生成质量、随机性和退化	Paper
⬜	Fast Inference from Transformers via Speculative Decoding	draft–verify 与拒绝采样；在保持目标分布不变的前提下并行生成多个 token	Paper
⬜	Accelerating Large Language Model Decoding with Speculative Sampling	speculative sampling 的独立推导与工程验证，适合和上一论文对照阅读	Paper
⬜	SpecInfer: Accelerating Large Language Model Serving with Tree-based Speculative Inference and Verification	用候选树提高并行验证宽度，连接解码算法与 serving batch	Paper
⬜	Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads	不依赖独立 draft model 的多头预测与 tree attention	Paper
⬜	Break the Sequential Dependency of LLM Inference Using Lookahead Decoding	从 Jacobi iteration 理解无额外模型的并行候选生成	Paper
⬜	EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty	在 feature space 自回归预测，理解高接受率 drafter 的训练方法	Paper
⬜	Better & Faster Large Language Models via Multi-token Prediction	训练时预测多个未来 token，将训练目标与推理解码并行性连接起来	Paper

General

Status	Paper	Venue	Links
✅	A Survey of LLM Inference Systems	—	Paper / Note
⬜	WaferLLM: Large Language Model Inference at Wafer Scale	OSDI'25	Paper

Long Context Inference

Status	Paper	Venue	Links
✅	Training-Free Long-Context Scaling of Large Language Models	ICML'24	Paper / Note
✅	Efficient Streaming Language Models with Attention Sinks	ICLR'24	Paper
✅	Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference	ICML'24	Paper
✅	DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads	ICLR'25	Paper
✅	MiniMax Sparse Attention	arXiv'26	Paper / Note
⬜	Unifying Sparse Attention with Hierarchical Memory for Scalable Long-Context LLM Serving (SPIN)	arXiv'26	Paper

LLM Serving

Status	Paper	Venue	Links
⬜	Orca: A Distributed Serving System for Transformer-Based Generative Models	OSDI'22	Paper
⬜	Efficient Memory Management for Large Language Model Serving with PagedAttention	SOSP'23	Paper
⬜	FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU	ICML'23	Paper
⬜	AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving	OSDI'23	Paper
⬜	FastServe: Fast Distributed Inference Serving for Large Language Models	arXiv'23	Paper
⬜	InferCept: Efficient Interleaving of Inference and Decoding for LLM Serving	ICML'24	Paper
⬜	Punica: Multi-Tenant LoRA Serving	MLSys'24	Paper
⬜	S-LoRA: Serving Thousands of Concurrent LoRA Adapters	MLSys'24	Paper
⬜	Splitwise: Efficient Generative LLM Inference Using Phase Splitting	ISCA'24	Paper
⬜	Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve	OSDI'24	Paper
⬜	Llumnix: Dynamic Scheduling for Large Language Model Serving	OSDI'24	Paper
⬜	Preble: Efficient Distributed Prompt Scheduling for LLM Serving	arXiv'24	Paper
✅	SGLang: Efficient Execution of Structured Language Model Programs	—	Paper
✅	FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving	—	Paper
⬜	DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving	OSDI'24	Paper
⬜	LoongServe: Efficiently Serving Long-Context Large Language Models with Elastic Sequence Parallelism	SOSP'24	Paper
⬜	Mooncake: Trading More Storage for Less Computation — A KVCache-centric Architecture for Serving LLM Chatbot	FAST'25	Paper
⬜	NanoFlow: Towards Optimal Large Language Model Serving Throughput	OSDI'25	Paper
⬜	From Tokens to Layers: Redefining Stall-Free Scheduling for LLM Serving with Layered Prefill	MLSys'26	Paper
⬜	CRAFT: Fine-Grained Cost-Aware Expert Replication for Efficient Mixture-of-Experts Serving	MLSys'26	Paper
⬜	Strata: Hierarchical Context Caching for Long Context Language Model Serving	OSDI'26	Paper
⬜	No Buffer, No Bottleneck: Efficient Zero-Copy KV Cache Offloading for Long-Context LLMs (DirectKV)	OSDI'26	Paper

Quantization & Compression

Status	Paper	Why It Matters	Links
⬜	LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale	混合精度分解 activation outlier，是理解 LLM INT8 的起点	Paper
⬜	GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers	基于近似二阶信息的 one-shot weight-only PTQ	Paper
⬜	SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models	用等价缩放把 activation 量化难度迁移到 weight，形成 W8A8 路线	Paper
⬜	AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration	用 activation 识别显著通道并保护关键权重，形成主流 W4A16 路线	Paper
⬜	SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression	将少量 outlier 与低比特权重分离，理解稀疏异常值处理	Paper
⬜	OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models	通过可学习的 clipping 与等价变换改善低比特 PTQ	Paper
⬜	AQLM: Extreme Compression of Large Language Models via Additive Quantization	additive codebook 与极低 bit/weight，连接压缩算法和解码 kernel	Paper
⬜	KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache	区分 K/V 的通道与 token 统计特性，理解 KV cache 低比特化	Paper
⬜	QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs	用 Hadamard rotation 消除 outlier，适合从算法追到融合 kernel	Paper
⬜	The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits	BitNet b1.58 与三值权重，理解量化感知训练的架构路线	Paper

MegaKernel

Status	Paper	Venue	Links
✅	Look Ma, No Bubbles! Designing a Low-Latency Megakernel for Llama-1B	Blog	Paper
✅	MPK: A Compiler and Runtime for Mega-Kernelizing Tensor Programs	OSDI'26	Paper
✅ 📝	Event Tensor: A Unified Abstraction for Compiling Dynamic Megakernel	MLSys'26	Paper / Note
✅	TileRT: Tile-Based Runtime for Ultra-Low-Latency LLM Inference	—	Paper
✅	SonicMoE: Accelerating MoE with IO and Tile-aware Optimizations	arXiv'25	Paper
✅ 📝	MegaMoE (fused MoE megakernel, DeepGEMM)	DeepGEMM PR	Source / Note
✅	Compiling LLMs into a MegaKernel: A Path to Low-Latency Inference	Blog	Paper
⬜	Ada-MK: Adaptive MegaKernel Optimization via Automated DAG-based Search for LLM Inference	arXiv'26	Paper

MegaMoE 摘要（非正式论文，见 DeepGEMM PR #304）：把 MoE 前向中分发、两层分组 GEMM、SwiGLU、合并压进单一持久化 CUDA 核；用对称显存布局与 NVLink 在核内做专家并行词元交换，并以波次调度、L1/L2 词元池上的细粒度到达计数 / 掩码，把通信与计算流水重叠；SM100 上按分发、TMA+MMA、尾声与合并划分线程束角色与寄存器预算。

🏋️ LLM Training

Training Numerics & Memory

Status	Paper	Why It Matters	Links
⬜	Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour	global batch、线性学习率缩放与 warmup；理解数据并行扩展为何会改变优化行为	Paper
⬜	Mixed Precision Training	FP16 master weights、loss scaling 与数值范围；连接 Tensor Core 吞吐和收敛	Paper
⬜	Training Deep Nets with Sublinear Memory Cost	activation checkpointing/rematerialization 的经典计算–显存交换	Paper
⬜	Reducing Activation Recomputation in Large Transformer Models	sequence parallelism 与 selective recomputation，解释 Megatron 的 activation 内存优化	Paper
⬜	GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection	对梯度做低秩投影以降低 optimizer state 和训练显存	Paper

Distributed Training

Status	Paper	Venue	Links
⬜	PyTorch Distributed: Experiences on Accelerating Data Parallel Training	VLDB'20	Paper
⬜	GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism	NeurIPS'19	Paper
⬜	PipeDream: Generalized Pipeline Parallelism for DNN Training	SOSP'19	Paper
⬜	Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism	arXiv'19	Paper
⬜	ZeRO: Memory Optimizations Toward Training Trillion Parameter Models	SC'20	Paper
⬜	ZeRO-Offload: Democratizing Billion-Scale Model Training	USENIX ATC'21	Paper
⬜	Memory-Efficient Pipeline-Parallel DNN Training	ICML'21	Paper
⬜	ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning	SC'21	Paper
⬜	Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM	SC'21	Paper
⬜	GSPMD: General and Scalable Parallelization for ML Computation Graphs	arXiv'21	Paper
⬜	Alpa: Automating Inter- and Intra-Operator Parallelism for Distributed Deep Learning	OSDI'22	Paper
⬜	DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models	arXiv'23	Paper
⬜	Ring Attention with Blockwise Transformers for Near-Infinite Context	ICLR'24	Paper
⬜	Oobleck: Resilient Distributed Training of Large Models Using Pipeline Templates	SOSP'23	Paper
⬜	MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs	NSDI'24	Paper
⬜	LoongTrain: Efficient Training of Long-Sequence LLMs with Head-Context Parallelism	—	Paper
✅ 📝	PithTrain: A Compact and Agent-Native MoE Training System	arXiv'26	Paper / Note / Code Guide

Mixture-of-Experts Training

Status	Paper	Why It Matters	Links
⬜	Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer	现代稀疏 MoE 的起点：top-k gate、稀疏激活与负载均衡	Paper
⬜	GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding	将 MoE、SPMD sharding 和大规模 Transformer 训练结合	Paper
⬜	BASE Layers: Simplifying Training of Large, Sparse Models	用 balanced assignment 避免额外负载均衡损失	Paper
⬜	GLaM: Efficient Scaling of Language Models with Mixture-of-Experts	大规模稀疏语言模型的质量、计算与能耗权衡	Paper
⬜	DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale	expert parallel、通信与 MoE inference/training 系统化设计	Paper
⬜	FasterMoE: Modeling and Optimizing Training of Large-Scale Dynamic Pre-Trained Models	动态路由下的 shadowing 与拓扑感知通信优化	Paper
⬜	Tutel: Adaptive Mixture-of-Experts at Scale	自适应并行、all-to-all 与 kernel 优化的完整 MoE 系统	Paper
⬜	MegaBlocks: Efficient Sparse Training with Mixture-of-Experts	将 token dropping 问题转成 block-sparse GEMM，最贴近算子视角	Paper

RL Training

Status	Paper	Venue	Links
✅	Seer: Online Context Learning for Fast Synchronous LLM Reinforcement Learning	arXiv'25	Paper

Fine-Tuning & Alignment

Status	Paper	Why It Matters	Links
⬜	Fine-Tuning Language Models from Human Preferences	将 reward model 与 PPO 用于语言模型偏好优化的早期完整方案	Paper
⬜	Learning to Summarize from Human Feedback	展示偏好数据、reward model 和 RL 在真实生成任务中的规模化	Paper
⬜	Prefix-Tuning: Optimizing Continuous Prompts for Generation	冻结主模型，仅训练可学习 prefix 的参数高效微调	Paper
⬜	The Power of Scale for Parameter-Efficient Prompt Tuning	soft prompt 与模型规模关系，适合理解 PEFT 的表达能力	Paper
⬜	LoRA: Low-Rank Adaptation of Large Language Models	冻结权重并注入低秩更新，连接矩阵秩、训练显存和多租户推理	Paper
⬜	Proximal Policy Optimization Algorithms	RLHF 所需的最低限度策略优化基础：ratio、advantage 与 clipping	Paper
⬜	Finetuned Language Models Are Zero-Shot Learners	FLAN 与 instruction tuning，说明任务混合如何产生泛化能力	Paper
⬜	Training Language Models to Follow Instructions with Human Feedback	InstructGPT 的 SFT → reward model → PPO 三阶段链路	Paper
⬜	Constitutional AI: Harmlessness from AI Feedback	self-critique、revision 与 RLAIF，把安全原则引入后训练	Paper
⬜	Self-Instruct: Aligning Language Models with Self-Generated Instructions	自生成、过滤和扩增指令数据的经典流程	Paper
⬜	The Flan Collection: Designing Data and Methods for Effective Instruction Tuning	系统研究任务混合、模板、CoT 数据与 instruction tuning 配方	Paper
⬜	QLoRA: Efficient Finetuning of Quantized LLMs	NF4、double quantization、paged optimizer 与 LoRA 的组合	Paper
⬜	LIMA: Less Is More for Alignment	少量高质量监督数据与大规模预训练知识之间的分工	Paper
⬜	Direct Preference Optimization: Your Language Model is Secretly a Reward Model	将显式 reward model + PPO 化为稳定的 pairwise classification loss	Paper
⬜	RRHF: Rank Responses to Align Language Models with Human Feedback	用候选排序损失统一多种反馈来源	Paper
⬜	A General Theoretical Paradigm to Understand Learning from Human Preferences	IPO 与偏好优化的理论视角，理解 DPO 类方法的过拟合和正则化	Paper
⬜	KTO: Model Alignment as Prospect Theoretic Optimization	只需 desirable/undesirable 标签的非成对偏好学习	Paper
⬜	ORPO: Monolithic Preference Optimization without Reference Model	将 SFT 与偏好约束合并，移除独立 reference model	Paper
⬜	SimPO: Simple Preference Optimization with a Reference-Free Reward	reference-free、长度归一化 reward 与 margin objective	Paper
⬜	DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models	GRPO、数学数据与可验证奖励，是 reasoning RL 的关键前置	Paper
⬜	DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning	大规模 reasoning RL、冷启动数据与蒸馏路线	Paper

Compute-Communication Overlap

Status	Paper	Venue	Links
✅	Flux: Fast Software-based Communication Overlap on GPUs through Kernel Fusion	—	Paper
✅	DeepEP: An Efficient Expert-Parallel Communication Library	—	Paper
⬜	Centauri: Enabling Efficient Scheduling for Communication-Computation Overlap in Large Model Training via Communication Partitioning	ASPLOS'24	Paper
⬜	Comet: Fine-grained Computation-communication Overlapping for Mixture-of-Experts	—	Paper
⬜	TileLink: Generating Efficient Compute-Communication Overlapping Kernels using Tile-Centric Primitives	MLSys'25	Paper
⬜	Triton-distributed: Programming Overlapping Kernels on Distributed AI Systems with the Triton Compiler	—	Paper
⬜	FlashOverlap: A Lightweight Design for Efficiently Overlapping Communication and Computation	EuroSys'25	Paper
⬜	TokenWeave: Efficient Compute-Communication Overlap for Distributed LLM Inference	MLSys'26	Paper
⬜	UEP: Portable Expert-Parallel Communication	OSDI'26	Paper

🧠 Deep Learning

Foundations & Optimization

Status	Paper	Why It Matters	Links
⬜	Learning Representations by Back-Propagating Errors	从局部算子 backward 上升到计算图链式法则、梯度流和 activation 保存	Paper
⬜	Understanding the Difficulty of Training Deep Feedforward Neural Networks	Xavier initialization 与方差传播，理解初始化为何影响深层网络稳定性	Paper
⬜	Dropout: A Simple Way to Prevent Neural Networks from Overfitting	经典正则化、train/eval 行为差异与随机 mask	Paper
⬜	Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift	对比 batch statistics 与 LayerNorm，理解同步 BN 和训练/推理差异	Paper
⬜	Deep Residual Learning for Image Recognition	残差连接与深层梯度传播，是 Transformer residual stream 的结构前置	Paper
⬜	Adam: A Method for Stochastic Optimization	一阶/二阶矩、bias correction 与 optimizer state 显存	Paper
⬜	Decoupled Weight Decay Regularization	区分 L2 regularization 与 AdamW 的 decoupled weight decay	Paper
⬜	Layer Normalization	单样本归一化、训练稳定性与 Transformer 中的 reduction/fusion	Paper
⬜	Root Mean Square Layer Normalization	省去 re-centering 的 RMSNorm，连接现代 LLM 结构与高效 kernel	Paper
⬜	Adafactor: Adaptive Learning Rates with Sublinear Memory Cost	对二阶矩做 factored approximation，理解 optimizer memory 优化	Paper
⬜	Large Batch Optimization for Deep Learning: Training BERT in 76 Minutes	LAMB 的 layer-wise scaling 与超大 batch 训练	Paper
⬜	Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer	μP 与跨模型规模超参数迁移，连接 scaling experiment 和训练配方	Paper
⬜	Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training	低成本二阶曲率估计与 per-coordinate clipping	Paper

Language Modeling, Tokenization & Modern LLMs

Status	Paper	Why It Matters	Links
⬜	A Neural Probabilistic Language Model	embedding、条件概率与 next-token language modeling 的早期完整形式	Paper
⬜	Sequence to Sequence Learning with Neural Networks	自回归分解、teacher forcing、EOS 与 beam search 的基础	Paper
⬜	Neural Machine Translation by Jointly Learning to Align and Translate	additive attention 与 encoder–decoder alignment，理解 Transformer 之前的问题	Paper
⬜	Neural Machine Translation of Rare Words with Subword Units	BPE、词表大小、序列长度和输出 softmax 成本之间的关系	Paper
⬜	SentencePiece: A Simple and Language Independent Subword Tokenizer and Detokenizer for Neural Text Processing	从 raw text 训练 BPE/unigram tokenizer，理解现代 tokenizer pipeline	Paper
⬜	Improving Language Understanding by Generative Pre-Training	GPT-1 与 decoder-only 预训练–微调范式	Paper
⬜	BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding	encoder-only、masked LM 与 pretrain–finetune 范式	Paper
⬜	Language Models are Unsupervised Multitask Learners	GPT-2、zero-shot transfer 与 WebText 数据路线	Paper
⬜	Language Models are Few-Shot Learners	GPT-3、in-context learning 与 decoder-only scaling	Paper
⬜	Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer	T5、span corruption、C4 与统一 text-to-text interface	Paper
⬜	GLU Variants Improve Transformer	SwiGLU/GEGLU 与 gated FFN，解释现代 LLM 中三路投影和逐元素乘	Paper
⬜	RoFormer: Enhanced Transformer with Rotary Position Embedding	RoPE 的旋转与相对位置语义，以及 Q/K 融合实现	Paper
⬜	Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation	ALiBi 与无需显式位置 embedding 的长度外推	Paper
⬜	PaLM: Scaling Language Modeling with Pathways	大规模 dense LLM、Pathways 并行与训练行为分析	Paper
⬜	LLaMA: Open and Efficient Foundation Language Models	将 RMSNorm、SwiGLU、RoPE、tokenizer、数据与训练配方串成现代 LLM	Paper
⬜	Llama 2: Open Foundation and Fine-Tuned Chat Models	预训练、SFT、RLHF、安全评测与 chat model 的完整技术报告	Paper
⬜	Mistral 7B	sliding-window attention、GQA 与 rolling buffer KV cache	Paper
⬜	DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models	shared expert、细粒度 expert segmentation 与稀疏计算配比	Paper
⬜	Mixtral of Experts	实用 sparse MoE LLM 的结构、路由和质量–计算权衡	Paper
⬜	OLMo: Accelerating the Science of Language Models	开放数据、训练代码、checkpoint、日志和评测的端到端案例	Paper

Scaling Laws, Data & Pretraining

Status	Paper	Why It Matters	Links
⬜	Scaling Laws for Neural Language Models	建立 loss 与参数量、数据量、训练算力之间的幂律关系	Paper
⬜	Training Compute-Optimal Large Language Models	Chinchilla scaling 与 compute-optimal 参数/token 配比	Paper
⬜	Scaling Data-Constrained Language Models	数据受限和重复 epoch 下的 scaling behavior	Paper
⬜	The Pile: An 800GB Dataset of Diverse Text for Language Modeling	多域预训练语料组成、治理与 benchmark contamination	Paper
⬜	Deduplicating Training Data Makes Language Models Better	去重对记忆、评测污染、训练效率和质量的影响	Paper
⬜	Data Selection for Language Models via Importance Resampling	DSIR：用目标分布重要性重采样选择预训练数据	Paper
⬜	DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining	将多域数据配比转化为 group DRO 优化问题	Paper
⬜	The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only	大规模网页过滤与去重的生产级案例	Paper
⬜	Textbooks Are All You Need	合成高质量数据、数据质量与小模型能力的交换关系	Paper
⬜	Dolma: An Open Corpus of Three Trillion Tokens for Language Model Pretraining Research	开放语料的来源、过滤、去重、PII 与治理流程	Paper
⬜	DataComp-LM: In Search of the Next Generation of Training Sets for Language Models	受控比较过滤、去重和数据混合策略	Paper
⬜	The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale	现代网页清洗、质量过滤与 FineWeb-Edu 配方	Paper

Attention Mechanisms & Variants

Status	Paper	Venue	Links
✅	Attention Is All You Need	NeurIPS'17	Paper / Note
✅	Big Bird: Transformers for Longer Sequences	NeurIPS'20	Paper / Note
✅	FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness	NeurIPS'22	Paper / Note
✅	FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning	arXiv	Paper / Note
✅	Flash-Decoding for Long-Context Inference	Blog	Paper / Note
✅	A Survey of Efficient Attention Methods: Hardware-efficient, Sparse, Compact, and Linear Attention	—	Paper

Sparse Attention Algorithms & Training

这一分类覆盖稀疏注意力本身的架构、选择算法与训练方法。MSA 和 DSA 是其中的现代案例，而不是分类边界。

Architecture and Modern Sparse Attention

Status	Paper	Why It Matters	Links
⬜	Fast Transformer Decoding: One Write-Head is All You Need	MQA 原始论文；理解所有 Query heads 共享 KV 与 decode 带宽瓶颈	Paper
⬜	GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints	建立 MHA–GQA–MQA 的连续关系；理解 MSA 为什么按 GQA group 独立选择	Paper
⬜	DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model	MLA、latent KV compression、decoupled RoPE，以及 uk_proj / uo_proj 的矩阵吸收	Paper
⬜	SeerAttention: Learning Intrinsic Sparse Attention in Your LLMs	可学习的 block gate、自蒸馏与 block-sparse kernel；最接近 MSA Indexer 的对照之一	Paper
⬜	Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention	同时研究压缩、选择、局部窗口与硬件对齐；理解 DSA 的算法–kernel 协同背景	Paper
⬜	MoBA: Mixture of Block Attention for Long-Context LLMs	MoE 风格的 block routing；适合对比 MSA 的 block max-pooling 与 group-specific selection	Paper
⬜	MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention	无需重新训练的动态稀疏 prefill；对比学习式 Indexer 与预设 attention pattern	Paper
⬜	SpargeAttn: Accurate Sparse Attention Accelerating Any Model Inference	两阶段在线过滤与 softmax-aware pruning；理解推理期稀疏化的另一条路线	Paper
⬜	BLASST: Dynamic BLocked Attention Sparsity via Softmax Thresholding	复用 online softmax 统计量跳过低贡献 blocks；无需训练并覆盖 MHA/GQA/MQA/MLA	Paper

现代架构阅读顺序： MQA → GQA → DeepSeek-V2/MLA → SeerAttention → Native Sparse Attention → MoBA → Quest → MInference → SpargeAttn → BLASST。

Algorithm and Training Foundations

这组论文用于从算子实现反向补齐稀疏注意力依赖的算法与训练概念：稀疏 pattern、内容路由、不可微 Top-k、知识蒸馏、Router 稳定性和稀疏归一化。

Sparse Pattern and Content Routing

Status	Paper	Why It Matters	Links
⬜	Generating Long Sequences with Sparse Transformers	固定 factorized sparse pattern 与早期 block-sparse kernel；理解“规则但不自适应”的稀疏性	Paper
⬜	Longformer: The Long-Document Transformer	local window + global token；理解局部先验和全局信息通路	Paper
⬜	Reformer: The Efficient Transformer	LSH attention、bucket、排序和 reversible layers；理解动态候选集带来的数据重排成本	Paper
⬜	Efficient Content-Based Sparse Attention with Routing Transformers	online k-means 内容路由；连接固定 sparse pattern 与 learned Indexer	Paper

Big Bird 已在 Attention Mechanisms & Variants 中记录并标为已读，不在这里重复计数。

Differentiable Top-k and Discrete Selection

Status	Paper	Why It Matters	Links
⬜	Sparser is Faster and Less is More: Efficient Sparse Attention for Long-Range Transformers	SparseK scoring network + differentiable Top-k；直接对比 MSA/DSA 的 hard Top-k + KL 路线	Paper
⬜	Differentiable Top-k Operator with Optimal Transport	用熵正则 Optimal Transport 平滑 Top-k，理解连续松弛及其梯度	Paper
⬜	Fast, Differentiable and Sparse Top-k: A Convex Analysis Perspective	从凸优化构造可微且真正稀疏的 Top-k，并讨论 GPU/TPU-friendly 算法	Paper
⬜	Categorical Reparameterization with Gumbel-Softmax	离散采样的经典连续松弛；理解 temperature、annealing 与 soft-train/hard-inference 差异	Paper

Distillation and Router Training

Status	Paper	Why It Matters	Links
✅ 📝	Distilling the Knowledge in a Neural Network	soft target、temperature 与 KL；理解主 attention 如何作为 Indexer teacher	Paper / Note
⬜	Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity	hard Top-1 routing、capacity 与 load-balancing loss；将 Indexer 理解成 memory router	Paper
⬜	ST-MoE: Designing Stable and Transferable Sparse Expert Models	Router 稳定性、辅助损失与 router z-loss；理解小型路由器如何影响整个模型训练	Paper
⬜	Mixture-of-Experts with Expert Choice Routing	expert 选择 token 而非 token 选择 expert；类比 q2k→k2q reverse index 与 KV-owner 调度	Paper

Sparse Normalization

Status	Paper	Why It Matters	Links
⬜	From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification	在概率 simplex 上产生精确零值并保留可计算 Jacobian	Paper
⬜	Adaptively Sparse Transformers	使用可学习的 $\alpha$-entmax 让不同 attention heads 自适应选择稠密或稀疏分布	Paper

算法/训练补课顺序： Sparse Transformer → Routing Transformer → SparseK Attention → Differentiable Top-k → Knowledge Distillation → Switch Transformer / ST-MoE → 回看 SeerAttention、NSA、MSA 与 DSA。

New Architectures

Status	Paper	Venue	Links
⬜	Hyena Hierarchy: Towards Larger Convolutional Language Models	ICML'23	Paper
⬜	RWKV: Reinventing RNNs for the Transformer Era	EMNLP'23	Paper
⬜	Retentive Network: A Successor to Transformer for Large Language Models	arXiv'23	Paper
⬜	Mamba: Linear-Time Sequence Modeling with Selective State Spaces	arXiv'23	Paper
⬜	Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality	ICML'24	Paper
⬜	xLSTM: Extended Long Short-Term Memory	NeurIPS'24	Paper
✅	Gated Linear Attention Transformers with Hardware-Efficient Training	arXiv	Paper / Note
✅	Kimi Linear Attention: An Expressive, Efficient Attention Architecture	arXiv'25	Paper
✅	DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models	arXiv'25	Paper
✅ 📝	DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence	Tech report'26	Paper / Note

On-Device / Mobile

Status	Paper	Venue	Links
✅	On-Device Training Under 256KB Memory	NeurIPS'22	Paper
✅	PockEngine: Sparse and Efficient Fine-tuning in a Pocket	MICRO'23	Paper / Note

📊 LLM Evaluation & Safety

Capability & Quality Evaluation

Status	Paper	Why It Matters	Links
⬜	Measuring Massive Multitask Language Understanding	MMLU：用多学科考试题衡量知识与问题求解能力	Paper
⬜	Beyond the Imitation Game: Quantifying and Extrapolating the Capabilities of Language Models	BIG-bench：异构任务与能力随规模变化	Paper
⬜	Holistic Evaluation of Language Models	HELM：统一准确率、校准、鲁棒性、公平性、毒性和效率	Paper
⬜	TruthfulQA: Measuring How Models Mimic Human Falsehoods	将事实真实性与普通知识准确率区分开	Paper
⬜	Training Verifiers to Solve Math Word Problems	GSM8K 与 verifier 路线，连接推理生成和可验证结果	Paper
⬜	Measuring Mathematical Problem Solving With the MATH Dataset	竞赛数学、分步推理与严格答案评测	Paper
⬜	Evaluating Large Language Models Trained on Code	HumanEval 与 pass@k，理解代码生成评测	Paper
⬜	GPQA: A Graduate-Level Google-Proof Q&A Benchmark	高难度、抗搜索污染的专家级科学问答	Paper
⬜	Instruction-Following Evaluation for Large Language Models	IFEval：用可验证约束衡量 instruction following	Paper
⬜	Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena	LLM judge、位置偏差、冗长偏差和 pairwise evaluation	Paper
⬜	Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference	真实用户盲测、成对偏好与 Elo/Bradley–Terry 排名	Paper
⬜	RewardBench: Evaluating Reward Models for Language Modeling	对话、推理、安全等场景的 reward model 基准	Paper
⬜	SWE-bench: Can Language Models Resolve Real-World GitHub Issues?	真实仓库、issue、patch 与测试驱动的软件工程评测	Paper
⬜	LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code	持续更新题目与时间切分，降低代码评测污染	Paper
⬜	Lessons from the Trenches on Reproducible Evaluation of Language Models	prompt、tokenizer、版本和实现细节对评测复现的影响	Paper

Safety, Robustness & Red Teaming

Status	Paper	Why It Matters	Links
⬜	RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models	开放式生成中的 toxicity 测量与 prompt 条件效应	Paper
⬜	Red Teaming Language Models with Language Models	用模型自动生成和筛选攻击，建立可扩展 red teaming	Paper
⬜	Universal and Transferable Adversarial Attacks on Aligned Language Models	GCG 对抗后缀与可迁移 jailbreak 的基础工作	Paper
⬜	XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models	同时衡量合理拒绝与过度拒绝	Paper
⬜	SafetyBench: Evaluating the Safety of Large Language Models	多类别、多语言的安全知识与行为评测	Paper
⬜	Towards Understanding Sycophancy in Language Models	研究模型迎合用户观点的表现及训练信号来源	Paper
⬜	Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training	研究条件触发的欺骗策略能否熬过安全训练	Paper
⬜	HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal	jailbreak、攻击方法与稳健拒绝的标准化评测	Paper
⬜	A StrongREJECT for Empty Jailbreaks	jailbreak 评测必须衡量实际危害、完成度和拒绝质量	Paper
⬜	The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning	高风险双用途知识评测及其与 unlearning 的关系	Paper

🤖 LLM for Kernel Optimization

Status	Paper	Venue	Links
✅	AVO: Agentic Variation Operators for Autonomous Evolutionary Search	arXiv'26	Paper / Note
✅ 📝	CAKE: Compiler-Agent Co-Design for Frontier Kernel Evolution	arXiv'26	Paper / Note
⬜	FlashInfer-Bench: Building the Virtuous Cycle for AI-driven LLM Systems	MLSys'26	Paper
⬜	Harness Engineering for LLM-Driven GPU Kernel Generation	arXiv'26	Paper
⬜	Agentic Kernel Optimization: Generating State-of-the-Art GPU Kernels Without Hand-Written CUDA	arXiv'26, preliminary	Paper

🧩 Agent Systems

Status	Paper	Venue	Links
✅ 📝	SkVM: Revisiting Language VM for Skills across Heterogenous LLMs and Harnesses	arXiv'26	Paper / Note

🖥️ GPU Microarchitecture

Status	Paper	Venue	Links
⬜	Understanding Latency Hiding on GPUs	—	Paper

📐 Math Foundations

Status	Paper	Venue	Links
⬜	Categorical Foundations for CuTe Layouts	—	Paper

⚙️ Compiler

Status	Paper	Venue	Links
✅	Honeycomb: Secure and Efficient GPU Executions via Static Validation	OSDI'23	Paper / Note
✅	HIDA: A Hierarchical Dataflow Compiler for High-Level Synthesis	ASPLOS'24	Paper / Note

🐧 Operating Systems

Status	Paper	Venue	Links
✅	RedLeaf: Isolation and Communication in a Safe Operating System	OSDI'20	Paper / Note
✅	Theseus: an Experiment in Operating System Structure and State Management	OSDI'20	Paper
✅	Unikraft: Fast, Specialized Unikernels the Easy Way	EuroSys'21	Paper / Note
✅	The Demikernel Datapath OS Architecture for Microsecond-scale Datacenter Systems	SOSP'21	Paper / Note

🛡️ Hypervisor & Virtualization

Status	Paper	Venue	Links
✅	HyperBench: A Benchmark Suite for Virtualization Capabilities	—	Paper / Note
✅	DuVisor: a User-level Hypervisor Through Delegated Virtualization	arXiv'22	Paper
✅	AvA: Accelerated Virtualization of Accelerators	ASPLOS'22	Paper
✅	Security and Performance in the Delegated User-level Virtualization	OSDI'23	Paper / Note
✅	System Virtualization for Neural Processing Units	HotOS'23	Paper
✅	Nephele: Extending Virtualization Environments for Cloning Unikernel-based VMs	EuroSys'23	Paper / Note
✅	Honeycomb: Secure and Efficient GPU Executions via Static Validation	OSDI'23	Paper / Note

🔬 RISC-V

Status	Paper	Venue	Links
✅	A First Look at RISC-V Virtualization from an Embedded Systems Perspective	TC'21	Paper
✅	CVA6 RISC-V Virtualization: Architecture, Microarchitecture, and Design Space Exploration	arXiv'23	Paper

---

If you find this list helpful, feel free to ⭐ star this repo!