HyperFix通过非线性修正实现多任务向量融合，无需重复调参。
HyperFix: Combinatorial Nonlinear Correction for Task Vector Merging
- 设计轻量超网络，根据任务子集预测权重空间的非线性修正
- 在多种基准上超越现有方法，且大子集融合无需额外调优
- 适合需要灵活组合多个任务模型的场景
任务向量可实现模型合并而无需联合训练。但实际中需合并的任务向量子集可能变化，现有方法通常对特定子集使用标量调参，需针对不同子集重复调参，限制了合并方式仅能线性缩放。为此，我们将跨不同任务子集的合并问题建模为组合校正问题，提出 HyperFix——一个轻量级超网络，用于预测基于子集条件的权重空间非线性修正。该模型在来自任务库的单个、成对和三元子集上训练一次，即可泛化至更大子集，无需针对每个子集进行优化。局部扰动分析界定了线性合并后的残差修正，并支持从微小任务更新中学习修正项。在多种基准上的实验表明，HyperFix 在性能上优于现有任务向量合并方法，同时显著降低调参成本。
原文摘要 · Abstract (English)
Task vectors enable model merging without joint retraining. In practice, the subset of task vectors to be merged may vary, but many existing methods use scalar tuning for a particular subset, requiring repeated tuning across subsets and restricting task vector merging to linear rescaling. We therefore formulate merging across varying task subsets as a combinatorial correction problem and introduce HyperFix, a lightweight hypernetwork that predicts subset-conditioned nonlinear corrections in weight space. Trained once on singleton, pair, and triple subsets from a task bank, HyperFix generalizes to larger subsets without per-subset optimization. Our local perturbation analysis bounds the residual correction beyond linear merging and motivates learning it from small task updates. Experiments across diverse benchmarks show that HyperFix outperforms existing task vector merging methods while reducing tuning cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。