不需训练，用低秩估计融合大模型，提升性能同时减少干扰。
LoRE-Merging: Exploring Low-Rank Estimation For Large Language Model Merging
- 基于任务向量的低秩估计，无需访问原始模型。
- 实验证明可有效降低模型融合时的干扰，保留任务特异性信息。
- 适合追求高效模型融合、无训练成本的研究者或工程师。
当前多数方法依赖微调或强化学习等进一步训练来提升模型能力，而模型融合则能在不需额外训练的情况下改进模型。本文提出一种基于任务向量低秩估计的统一融合框架——LoRE-Merging，无需访问基础模型即可实现融合。该方法基于观察：微调模型的任务向量常具有少数显著奇异值，因此低秩估计更不易受干扰。通过将融合问题建模为优化问题，实验表明该框架能有效缓解干扰并保留任务特定信息，显著提升模型融合的现有技术水平。
原文摘要 · Abstract (English)
While most current approaches rely on further training techniques, such as fine-tuning or reinforcement learning, to enhance model capacities, model merging stands out for its ability of improving models without requiring any additional training. In this paper, we propose a unified framework for model merging based on low-rank estimation of task vectors without the need for access to the base model, named \textsc{LoRE-Merging}. Our approach is motivated by the observation that task vectors from fine-tuned models frequently exhibit a limited number of dominant singular values, making low-rank estimations less prone to interference. We implement the method by formulating the merging problem as an optimization problem. Extensive empirical experiments demonstrate the effectiveness of our framework in mitigating interference and preserving task-specific information, thereby advancing the state-of-the-art performance in model merging techniques.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。