Miles v0.1打造可生产级的强化学习后训练系统，支持大模型全流程高效迭代。
Miles v0.1: Production-Level Post-Training
- 基于SGLang构建可验证的滚动引擎，模块化设计支持灵活定制
- 64块GB300 GPU上实现744B模型异步智能体训练，单步中位耗时263秒
- 开源支持全参数与LoRA强化学习，适合研究者和企业快速部署
我们提出Miles v0.1，一个面向前沿强化学习后训练的完整栈生产级系统。基于slime的清晰架构，Miles在强化学习训练循环的每个阶段均遵循单一原则：组件必须可验证、干净且可定制。以准确性、效率、可靠性和可扩展性为核心目标，使前沿规模强化学习对研究人员和企业都可及。本报告详述系统全链路：基于SGLang的滚动引擎、支持NVIDIA Megatron-LM与PyTorch FSDP的两种后端训练器，以及适用于不同部署拓扑的三种权重同步传输方式。除全参数强化学习外，还支持LoRA强化学习、在线策略蒸馏、监督微调与真正在线策略滚动-训练对齐，并将同一架构扩展至扩散模型。最后展示端到端案例：在64块NVIDIA GB300 GPU上，对GLM-5.2 744B-A40B模型进行终端编码任务的完全异步智能体强化学习，前30个步骤的中位单步耗时为263秒。项目开源地址：https://github.com/radixark/miles，官网：https://miles.radixark.com。
原文摘要 · Abstract (English)
We present Miles v0.1, a full-stack, production-ready system for frontier post-training. Building upon the clean design of slime, Miles designs each stage of the reinforcement-learning (RL) training loop around a single principle: components should be verified, clean, and customizable. With accuracy, efficiency, reliability, and scalability as first-class goals, Miles aims to make frontier-scale RL accessible to researchers and enterprises alike. This report walks through the system end to end: rollout engines built on SGLang, a trainer with a choice of two backends (NVIDIA Megatron-LM and PyTorch FSDP), and three weight-synchronization transports for different deployment topologies. Beyond full-parameter RL, Miles also supports LoRA RL, on-policy distillation, supervised fine-tuning, and true-on-policy rollout-training alignment, and extends the same architecture to diffusion models. We close with an end-to-end case study: fully asynchronous agentic RL on a GLM-5.2 744B-A40B model over terminal-use coding tasks, running on 64 NVIDIA GB300 GPUs with a median step time of 263 seconds over the first 30 measured steps. Miles is open-sourced at https://github.com/radixark/miles, with the project website at https://miles.radixark.com.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。