arXiv:2509.18208cs.LGcs.AI2025-09NeurIPS被引 1

通过变分推理优化任务向量组合,实现更稳定高效的多任务知识融合。

Variational Task Vector Composition

论文配图:Variational Task Vector Composition
图 1 · 摘自论文原文
  • 将组合系数设为潜在变量,用贝叶斯框架进行样本级动态调整。
  • 在多个数据集上超越现有方法,通过筛选可靠组件提升性能。
  • 引入稀疏先验与门控采样,提高可解释性与泛化能力,适合高效多任务学习者。

任务向量通过记录预训练模型与特定任务权重之间的差异,捕捉微调过程中的变化。任务向量的组合是任务算术中的关键操作,使模型能在不增加推理开销的情况下整合多个任务的知识。本文提出变分任务向量组合,将组合系数视为潜在变量,并在贝叶斯推断框架下进行估计。不同于以往在任务层面操作的方法,本框架聚焦于样本级别的组合。受任务向量中结构冗余现象的启发,我们引入了Spike-and-Slab先验,促进稀疏性并仅保留最具有信息量的成分。为进一步缓解稀疏高维空间中的高方差与采样效率低的问题,我们设计了一种门控采样机制,基于不确定性和重要性过滤组合系数,构建可控后验分布。该方法通过确定性选择可靠的组件,降低了采样方差,提升了透明度与泛化性能。实验结果表明,我们的方法在所有数据集上均优于现有方法,通过有选择地利用任务向量中最可靠、最具信息量的成分,展现出显著优势,确立了高效且有效任务向量组合的新标准。

原文摘要 · Abstract (English)

Task vectors capture how a model changes during fine-tuning by recording the difference between pre-trained and task-specific weights. The composition of task vectors, a key operator in task arithmetic, enables models to integrate knowledge from multiple tasks without incurring additional inference costs. In this paper, we propose variational task vector composition, where composition coefficients are taken as latent variables and estimated in a Bayesian inference framework. Unlike previous methods that operate at the task level, our framework focuses on sample-specific composition. Motivated by the observation of structural redundancy in task vectors, we introduce a Spike-and-Slab prior that promotes sparsity and preserves only the most informative components. To further address the high variance and sampling inefficiency in sparse, high-dimensional spaces, we develop a gated sampling mechanism that constructs a controllable posterior by filtering the composition coefficients based on both uncertainty and importance. This yields a more stable and interpretable variational framework by deterministically selecting reliable task components, reducing sampling variance while improving transparency and generalization. Experimental results demonstrate that our method consistently outperforms existing approaches across all datasets by selectively leveraging the most reliable and informative components in task vectors. These findings highlight the practical value of our approach, establishing a new standard for efficient and effective task vector composition.

任务向量变分推断多任务学习稀疏性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。