Alchemist通过复用服务阶段激活值，显著提升大模型持续学习的训练效率。
Alchemist: Towards the Design of Efficient Online Continual Learning System
- 只在预填充阶段记录激活值和键值缓存，降低延迟与内存开销。
- 通过智能激活卸载与冗余规避，训练吞吐量最高提升1.72倍。
- 适合需要高效在线更新的大模型系统，尤其关注低延迟部署场景。
持续学习为利用用户反馈增量优化大语言模型提供了有前景的解决方案。其中，在线持续学习——通过小批次用户反馈迭代训练模型——已展现出显著性能提升。然而，现有方法将训练与服务过程分离，导致在线训练器需重复计算服务阶段已生成的中间结果，冗余计算占总训练时间的30%至42%。本文提出Alchemist，据我们所知，首个能高效复用服务激活值以提升训练吞吐量的在线持续学习系统。Alchemist引入两项关键技术：(1) 仅在预填充阶段记录并存储激活值与键值缓存，以最小化延迟和内存开销；(2) 智能激活卸载与冗余规避。基于ShareGPT数据集采样的不同长度输入进行评估显示，相比独立训练集群，Alchemist可使训练吞吐量最高提升1.72倍，训练期间内存使用减少达47%，支持最多两倍的训练词元数量，同时对服务延迟影响可忽略不计。
原文摘要 · Abstract (English)
Continual learning has become a promising solution to refine large language models incrementally by leveraging user feedback. In particular, online continual learning - iteratively training the model with small batches of user feedback - has demonstrated notable performance improvements. However, the existing practice of separating training and serving processes forces the online trainer to recompute the intermediate results already done during serving. Such redundant computations can account for 30%-42% of total training time. In this paper, we propose Alchemist, to the best of our knowledge, the first online continual learning system that efficiently reuses serving activations to increase training throughput. Alchemist introduces two key techniques: (1) recording and storing activations and KV cache only during the prefill phase to minimize latency and memory overhead; and (2) smart activation offloading and hedging. Evaluations with inputs of varied token length sampled from ShareGPT dataset show that compared with a separate training cluster, Alchemist significantly increases training throughput by up to 1.72x, reduces up to 47% memory usage during training, and supports up to 2x more training tokens - all while maintaining negligible impact on serving latency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。