Molt让智能体强化学习训练更简单，代码清晰易改。
Molt: A Scalable PyTorch-Native Training Framework for Agentic Reinforcement Learning
- 用PyTorch原生设计，代码简洁可读，研究者能全程追踪算法流
- 异步训练多模态与专家混合策略，不使用自己未生成的令牌
- 性能媲美顶尖框架，适合想快速实验的强化学习研究者
智能体强化学习研究常需频繁修改算法、新估计器、新流水线阶段和新回滚方案，主流框架中每次改动都需在训练器、分布式后端和回滚胶水层间穿行，研究成本高。Molt是专为降低此成本设计的PyTorch原生训练框架：代码紧凑干净，研究者可整脑记忆，AI编码助手亦可完整理解，实现从头到尾的算法追踪与修改。智能体即普通程序，一个异步循环同时训练多模态与专家混合策略，且始终不使用自身未生成的令牌，确保令牌、策略版本与模型语义一致。精简设计不影响性能：在匹配的全异步协议下，Molt统计上可媲美最先进基于Megatron的系统。Molt已开源，提供配方与容器，详见https://github.com/NVIDIA-NeMo/labs-molt。
原文摘要 · Abstract (English)
Agentic reinforcement learning research is constant algorithm modification, new estimators, new pipeline stages, new rollout schemes, and in mainstream frameworks each change threads through layers of trainer, distributed backend, and rollout glue: the cost lands on the researcher at every iteration. Molt is a PyTorch-native training framework built to keep that cost small: a codebase compact and clean enough for a researcher to hold in their head, and for an AI coding assistant to read and reason about in its entirety, so the algorithm flow can be traced and changed end to end. The agent is an ordinary program, and one asynchronous loop trains multimodal and mixture-of-experts policies while never training on a token it did not generate, consistent in tokens, policy versions, and model semantics. Leanness does not cost performance: under a matched, fully asynchronous protocol, Molt is statistically comparable to a state-of-the-art Megatron-based stack. Molt is open source and provides recipes and containers at https://github.com/NVIDIA-NeMo/labs-molt.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。