用数学公式一步生成可调控的模型融合结果，速度快且精准。
From Parameter to Representation: A Closed-Form Approach for Controllable Model Merging
- 将模型融合从参数优化转为直接修正最终表示，求解闭式解。
- 在多任务下生成更优的帕累托前沿，偏好对齐更精确。
- 计算复杂度线性增长，适合快速生成个性化模型。
模型融合旨在结合多个专家模型以实现多任务性能，但面临参数干扰问题。近年来兴起可控模型融合，使用户能显式平衡性能权衡。现有方法采用编译-查询范式，需进行昂贵的离线多目标优化，以实现快速、偏好感知的模型生成。该离线阶段通常涉及迭代搜索或专用训练，其复杂度随任务数量呈指数增长。为克服此限制，我们从参数空间优化转向直接修正模型最终表示。本方法将修正建模为最优线性变换，获得闭式解，从而以单步、架构无关的计算取代整个离线优化过程。该解直接融入用户偏好，可在运行时快速生成帕累托最优模型，复杂度仅随任务数线性增长。实验表明，本方法生成的帕累托前沿更优，偏好对齐更精确，计算成本显著降低。
原文摘要 · Abstract (English)
Model merging combines expert models for multitask performance but faces challenges from parameter interference. This has sparked recent interest in controllable model merging, giving users the ability to explicitly balance performance trade-offs. Existing approaches employ a compile-then-query paradigm, performing a costly offline multi-objective optimization to enable fast, preference-aware model generation. This offline stage typically involves iterative search or dedicated training, with complexity that grows exponentially with the number of tasks. To overcome these limitations, we shift the perspective from parameter-space optimization to a direct correction of the model's final representation. Our approach models this correction as an optimal linear transformation, yielding a closed-form solution that replaces the entire offline optimization process with a single-step, architecture-agnostic computation. This solution directly incorporates user preferences, allowing a Pareto-optimal model to be generated on-the-fly with complexity that scales linearly with the number of tasks. Experimental results show our method generates a superior Pareto front with more precise preference alignment and drastically reduced computational cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。