用进化策略训练语言模型，显著降低内存占用和推理体积。
EOE: Evolutionary Optimization of Experts for Training Language Models
- 每次只训练一个专家，通过进化操作融合最优专家经验。
- 最佳专家精度接近全模型，推理模型体积大幅缩减。
- 纯C++/CUDA实现，适合在电脑和边缘设备部署。
本文提出一种用于大语言模型训练的进化框架。模型被划分为多个结构相同但参数不同的专家（子网络），每个训练步骤仅更新一个专家。在经典AdamW优化后，引入交叉、粒子群优化（PSO）和变异等进化算子，将当前专家与最优专家的权重进行融合，使当前专家能学习最优方向，加速损失下降。最终仅保存最优专家的权重。实验表明，最优专家的精度几乎与全模型相当，显著减少推理模型尺寸。由于每步仅训练一个专家，训练所需内存大幅降低，吞吐量提升超过十倍。代码为纯C++/CUDA实现，便于在个人电脑和边缘计算设备上部署。
原文摘要 · Abstract (English)
This paper presents an evolutionary framework for the training of large language models(LLM). The models are divided into several experts(sub-networks), which have the same structure but different parameter values. Only one expert is trained at each step. After the classical AdamW optimization, some evolutionary operators(crossover, PSO, and mutation) act on the tensor weights between the current expert and the best expert. So current expert would learn the experience of best expert. The direction of best expert would help current expert's loss decrease faster. Finally, only save the weight of the best expert. Experiments show that best expert would achieve nearly the same accuracy as the full model. This would greatly reduce the size of the model for inference. Since only one expert is trained at each step, the training needs much less memory and has much higher throughput. Experiments show that the throughput would accelerate more than ten times! Our source code is available. It's a pure c++/cu framework, which is suitable for easy deployment on PCs and edge computing devices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。