PithTrain让代码代理更高效地训练MoE模型，减少6成操作次数。
PithTrain: A Compact and Agent-Native MoE Training System
- 专为代码代理设计的轻量级MoE训练框架
- 吞吐量媲美生产级系统，代理操作减少62%
- 适合希望自动化训练框架开发的研究者
混合专家（MoE）已成为前沿语言模型的主流架构。为满足这一需求，生产级框架经过多年的工程积累，构建了优化的MoE训练栈。然而，针对新架构和系统优化更新这些栈仍成本高昂。随着AI代码代理的兴起，它们可自动化部分训练框架开发，加速演进。但将代理应用于现有框架存在隐性成本，当前仅以吞吐量评估无法察觉。我们提出代理任务效率（ATE）这一新维度：使用代码代理理解、操作和扩展框架的成本。基于四项代理原生设计原则，我们构建了紧凑、代理友好的PithTrain MoE训练框架，并引入ATE-Bench评测基准，覆盖真实训练框架任务。评估显示，PithTrain在吞吐量上与生产级框架相当，在ATE-Bench上实现更高代理任务效率，最多减少62%的代理交互轮次和64%的活跃GPU时间。
原文摘要 · Abstract (English)
Mixture-of-Experts (MoE) has become the dominant architecture for frontier language models. To meet this demand, production frameworks have built optimized MoE training stacks over years of engineering effort. Yet evolving these stacks for new architectures and system optimizations remains expensive. With the rise of AI coding agents, they could automate parts of training-framework development and accelerate this evolution. But applying them to these existing frameworks carries hidden costs, invisible to today's throughput-only evaluations. We name this missing dimension agent-task efficiency (ATE): the cost of using coding agents to understand, operate, and extend a framework. Grounded in four agent-native design principles, we build PithTrain, a compact, agent-native MoE training framework. We further introduce ATE-Bench, covering real-world training-framework tasks. Our evaluation shows PithTrain matches the throughput of production frameworks, and on ATE-Bench, PithTrain enables higher agent-task efficiency, with up to 62% fewer Agent Turns and 64% less Active GPU Time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。