AXLearn让大模型训练更灵活高效，支持跨硬件快速部署。
AXLearn: Modular, Hardware-Agnostic Large Model Training
- 模块化设计，组件间严格封装，便于快速组合与实验。
- 扩展时复杂度恒定，集成新功能仅需10行代码。
- 适合需要跨硬件部署的工业级大模型研发团队。
AXLearn 是一个用于大规模深度学习模型可扩展、高性能训练的生产系统。相较于其他先进深度学习系统，其独特优势在于模块化设计与对硬件无关训练的支持。系统内部软件组件间遵循严格的封装原则，使不同组件可灵活组合，加速模型开发与跨硬件平台的实验。相比现有系统中线性或二次复杂度的增长，AXLearn 在扩展组件时保持恒定复杂度。例如，在数百个模块中集成旋转位置编码（RoPE）仅需10行代码，而其他系统需数百行。同时，其性能与当前最先进训练系统相当。最后，本文分享了在苹果公司开发与运维 AXLearn 的实践经验。
原文摘要 · Abstract (English)
AXLearn is a production system which facilitates scalable and high-performance training of large deep learning models. Compared to other state-of-art deep learning systems, AXLearn has a unique focus on modularity and support for hardware-agnostic training. AXLearn's internal interfaces between software components follow strict encapsulation, allowing different components to be assembled to facilitate rapid model development and experimentation on different hardware infrastructure. AXLearn maintains constant complexity as we scale the components in the system, compared to linear or quadratic complexity in state-of-the-art training systems. This allows integrating features such as Rotary Position Embeddings (RoPE) into AXLearn across hundred of modules with just 10 lines of code, compared to hundreds as required in other systems. At the same time, AXLearn maintains equivalent performance compared to state-of-the-art training systems. Finally, we share our experience in the development and operation of AXLearn at Apple.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。