Contact emails
mao.cheng@radixark.ai,shi.dong@radixark.ai,banghua.zhu@radixark.ai
Project summary
Miles is a high-performance, open-source RL framework for large-scale model post-training, with novel systems work for trillion-parameter MoE stability and unified low-precision training-and-inference.
Project description
Miles is an enterprise-grade reinforcement learning framework specifically optimized for large-scale model post-training. It is built on top of PyTorch and ships the low-level systems work that makes RL stable and reproducible at trillion-parameter scale.
Key capabilities:
- Unified low-precision training — BF16, FP8, MXFP8, and INT4 W4A16 QAT recipes share a single quantization path between rollout and trainer so the policies are numerically aligned. An NVFP4 recipe is in development.
- Stable large-scale MoE RL — Rollout Routing Replay captures expert routing during inference and replays it during the trainer forward pass, reducing the routing mismatch that can destabilize large MoE RL training.
- Efficient rollout generation — Speculative rollout with online MTP-SFT keeps draft-model MTP layers aligned with the evolving actor policy, helping maintain decoding efficiency throughout training.
- Flexible post-training workflows — Miles supports LoRA-based SFT and RL, direct adapter loading into the rollout engine, and native agentic rollout for tool use, multi-turn dialogue, search, code execution, and multi-agent co-evolution.
- Extensible system design — More than twenty plug-points allow teams to replace rollout, reward, loss, filtering, or other components without forking the trainer.
- Ecosystem integration — Miles is designed to work with ecosystem partners such as Megatron, PyTorch, and Ray, making it easier to plug into existing large-scale training and orchestration stacks.
- Broad model and hardware support — Miles ships ready-to-run recipes for frontier and open-source models including DeepSeek-V4, Kimi K2 / K2.5 / K2.6, GLM-5 / 5.1, and Qwen3.5 / 3.6, with support for NVIDIA Hopper / Blackwell and AMD MI300-series GPUs via ROCm.
Are there any other projects in the PyTorch Ecosystem similar to yours? If, yes, what are they?
The closest existing PyTorch ecosystem project is TorchRL (https://github.com/pytorch/rl), which provides reusable RL primitives — environments, replay buffers, losses, transforms. TorchRL is intentionally a research-oriented building-block library; it does not target post-training of frontier LLMs, doesn't ship rollout/training disaggregation, doesn't integrate with high-throughput inference engines, and doesn't provide pre-tuned recipes for trillion-parameter MoE models.
Outside the official PyTorch Ecosystem, the projects that overlap with Miles in scope are open-source LLM post-training frameworks — TRL (Hugging Face), OpenRLHF, NVIDIA NeMo-Aligner, ByteDance veRL, and Tsinghua's slime. Compared to these, Miles emphasizes three areas: (1) MoE training stability via techniques like Rollout Routing Replay, which Miles introduced together with SGLang; (2) a unified low-precision path so rollout and trainer share the same FP8 / MXFP8 / INT4 quantization; (3) production-grade fault tolerance — rank-level recovery and RDMA-based P2P weight sync — designed for week-long training runs.
Miles is also designed to complement, rather than replace, major ecosystem partners. It builds on PyTorch, integrates with Megatron-style large-scale training, works with Ray-based orchestration, and connects to high-throughput rollout engines such as SGLang. This makes Miles a systems layer for LLM post-training that bridges PyTorch-based training, distributed orchestration, and inference-time rollout infrastructure.
Project repo URL
https://github.com/radixark/miles
Additional repos in scope of the application
Project license
Apache License 2.0 (see https://github.com/radixark/miles/blob/main/LICENSE).
GitHub handles of the project maintainer(s)
yueming-yuan,fzyzcjy,maocheng23,yushengsu-thu,Zhichenzzz,guapisolo,jybsuper,Shi-Dong,BHZ-BER,Ying1123
Is there a corporate or academic entity backing this project? If so, please provide the name and URL of the entity.
RadixArk (https://radixark.ai).
Website URL
https://www.radixark.com/miles/docs
Documentation
The full documentation site is hosted at https://www.radixark.com/miles/docs. It covers:
- Getting Started — installation (Docker, source, AMD), quick-start training run.
- User Guide — core concepts, argument groups, training-script walkthrough, monitoring, customization, rollout endpoints, fully-async rollout, agentic chat templates, full CLI reference.
- Per-model recipes — DeepSeek, Qwen, GLM, Kimi, Nemotron, MiMo, GPT-OSS.
- Advanced Features — Low Precision RL (FP8 / MXFP8 / INT4 QAT), INT4 QAT, R3, Speculative Decoding, LoRA, P2P weight transfer, fault tolerance, backends beyond Megatron.
- Developer Guide — architecture overview, debugging, contributing guide.
- FAQ.
How do you build and test the project today (continuous integration)? Please describe.
Miles uses GitHub Actions for continuous integration. Each pull request runs a multi-stage pipeline: fast CPU-only unit tests on GitHub-hosted runners, followed by per-recipe end-to-end pytest suites on self-hosted 8-GPU runners covering the dense and MoE training paths. Style and lint checks (ruff, black, isort) run as a pre-commit step on every PR. Separate workflows build and publish the official Docker images (NVIDIA cu129/cu130, AMD ROCm) and publish documentation per release. The local dev loop mirrors CI: pip install -r requirements.txt && pip install -e . --no-deps, then pytest tests/<area>.
Version of PyTorch
Miles tracks the PyTorch version shipped with its SGLang base image. The default Docker base is lmsysorg/sglang:v0.5.10 (PyTorch 2.6.x with CUDA 12.9 / 13.0). Specifically:
- NVIDIA cu129 x86 (default): PyTorch ≥ 2.6, CUDA 12.9.
- NVIDIA cu130 arm64 (GB300 image): PyTorch ≥ 2.6, CUDA 13.0.
- AMD ROCm 6.3+ image: PyTorch ≥ 2.6.
CI exercises the same image set per build.
Components of PyTorch
Miles is built on PyTorch end-to-end. Key components used directly:
torch.distributed — NCCL / RCCL collectives, process group setup, rank-aware checkpointing.torch.compile and the Inductor backend — used in the training path.- PyTorch native dtypes —
torch.bfloat16, torch.float8_e4m3fn, and torch.float8_e5m2 for unified BF16 / FP8 training and rollout.
torch.utils.data — dataset and rollout-buffer plumbing.torch.cuda.graphs — CUDA graph capture in the rollout-side execution path.
How long do you expect to maintain the project?
Miles is backed by Radixark, which is committed to maintaining the project indefinitely. The project also has contributors from the entire community ecosystem, including Nvidia, Meta, Humans& etc.
Additional information
No response
Contact emails
mao.cheng@radixark.ai,shi.dong@radixark.ai,banghua.zhu@radixark.ai
Project summary
Miles is a high-performance, open-source RL framework for large-scale model post-training, with novel systems work for trillion-parameter MoE stability and unified low-precision training-and-inference.
Project description
Miles is an enterprise-grade reinforcement learning framework specifically optimized for large-scale model post-training. It is built on top of PyTorch and ships the low-level systems work that makes RL stable and reproducible at trillion-parameter scale.
Key capabilities:
Are there any other projects in the PyTorch Ecosystem similar to yours? If, yes, what are they?
The closest existing PyTorch ecosystem project is TorchRL (https://github.com/pytorch/rl), which provides reusable RL primitives — environments, replay buffers, losses, transforms. TorchRL is intentionally a research-oriented building-block library; it does not target post-training of frontier LLMs, doesn't ship rollout/training disaggregation, doesn't integrate with high-throughput inference engines, and doesn't provide pre-tuned recipes for trillion-parameter MoE models.
Outside the official PyTorch Ecosystem, the projects that overlap with Miles in scope are open-source LLM post-training frameworks — TRL (Hugging Face), OpenRLHF, NVIDIA NeMo-Aligner, ByteDance veRL, and Tsinghua's slime. Compared to these, Miles emphasizes three areas: (1) MoE training stability via techniques like Rollout Routing Replay, which Miles introduced together with SGLang; (2) a unified low-precision path so rollout and trainer share the same FP8 / MXFP8 / INT4 quantization; (3) production-grade fault tolerance — rank-level recovery and RDMA-based P2P weight sync — designed for week-long training runs.
Miles is also designed to complement, rather than replace, major ecosystem partners. It builds on PyTorch, integrates with Megatron-style large-scale training, works with Ray-based orchestration, and connects to high-throughput rollout engines such as SGLang. This makes Miles a systems layer for LLM post-training that bridges PyTorch-based training, distributed orchestration, and inference-time rollout infrastructure.
Project repo URL
https://github.com/radixark/miles
Additional repos in scope of the application
sglang-milesbranch and contributes upstream.Project license
Apache License 2.0 (see https://github.com/radixark/miles/blob/main/LICENSE).
GitHub handles of the project maintainer(s)
yueming-yuan,fzyzcjy,maocheng23,yushengsu-thu,Zhichenzzz,guapisolo,jybsuper,Shi-Dong,BHZ-BER,Ying1123
Is there a corporate or academic entity backing this project? If so, please provide the name and URL of the entity.
RadixArk (https://radixark.ai).
Website URL
https://www.radixark.com/miles/docs
Documentation
The full documentation site is hosted at https://www.radixark.com/miles/docs. It covers:
How do you build and test the project today (continuous integration)? Please describe.
Miles uses GitHub Actions for continuous integration. Each pull request runs a multi-stage pipeline: fast CPU-only unit tests on GitHub-hosted runners, followed by per-recipe end-to-end pytest suites on self-hosted 8-GPU runners covering the dense and MoE training paths. Style and lint checks (
ruff,black,isort) run as a pre-commit step on every PR. Separate workflows build and publish the official Docker images (NVIDIA cu129/cu130, AMD ROCm) and publish documentation per release. The local dev loop mirrors CI:pip install -r requirements.txt && pip install -e . --no-deps, thenpytest tests/<area>.Version of PyTorch
Miles tracks the PyTorch version shipped with its SGLang base image. The default Docker base is
lmsysorg/sglang:v0.5.10(PyTorch 2.6.x with CUDA 12.9 / 13.0). Specifically:CI exercises the same image set per build.
Components of PyTorch
Miles is built on PyTorch end-to-end. Key components used directly:
torch.distributed— NCCL / RCCL collectives, process group setup, rank-aware checkpointing.torch.compileand the Inductor backend — used in the training path.torch.bfloat16,torch.float8_e4m3fn, andtorch.float8_e5m2for unified BF16 / FP8 training and rollout.torch.utils.data— dataset and rollout-buffer plumbing.torch.cuda.graphs— CUDA graph capture in the rollout-side execution path.How long do you expect to maintain the project?
Miles is backed by Radixark, which is committed to maintaining the project indefinitely. The project also has contributors from the entire community ecosystem, including Nvidia, Meta, Humans& etc.
Additional information
No response