用梯度法快速合并多个任务模型，性能接近全量微调。
SuperMerge: An Approach For Gradient-Based Model Merging
- 基于梯度的模型合并方法，无需重新训练。
- 合并后模型在多任务上表现接近全量微调结果。
- 支持分层合并，降低内存占用，适合部署场景。
大型语言模型如ChatGPT、Claude或LLaMA虽然功能强大，但体积庞大，难以满足低延迟和低成本的高吞吐应用需求。当已有模型需新增任务时，传统方法需对旧任务和新任务重新微调，成本高昂。为此，本文提出一种基于梯度的模型合并方法SUPERMERGE，可高效合并多个已针对不同任务微调的模型。该方法轻量快速，合并后的模型在所有任务上的表现与全量微调模型相当。此外，我们设计了一种分层合并策略，在不牺牲性能的前提下显著降低峰值内存占用。实验表明，SUPERMERGE在常见自然语言处理和计算机视觉任务中优于现有模型合并方法。
原文摘要 · Abstract (English)
Large language models, such as ChatGPT, Claude, or LLaMA, are gigantic, monolithic, and possess the superpower to simultaneously support thousands of tasks. However, high-throughput applications often prefer smaller task-specific models because of their lower latency and cost. One challenge of using task-specific models is the incremental need for solving newer tasks after the model is already deployed for existing tasks. A straightforward solution requires fine-tuning the model again for both existing and new tasks, which is computationally expensive and time-consuming. To address this issue, we propose a model merging based approach called SUPERMERGE. SUPERMERGE is a gradient-based method to systematically merge several fine-tuned models trained on existing and new tasks. SUPERMERGE is designed to be lightweight and fast, and the merged model achieves similar performance to fully fine-tuned models on all tasks. Furthermore, we proposed a hierarchical model merging strategy to reduce the peak space requirement without sacrificing the performance of the merged model. We experimentally demonstrate that SUPERMERGE outperforms existing model merging methods on common natural language processing and computer vision tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。