无需数据即可实现任务向量解耦，提升模型适配的稳定性和可扩展性。
Dataless Weight Disentanglement in Task Arithmetic via Kronecker-Factored Approximate Curvature
- 利用克罗内克分解近似曲率矩阵，构建无需外部数据的正则化方法。
- 在任务加法与反向操作中达到当前最优性能，且计算复杂度恒定。
- 适合隐私敏感或无任务数据场景下的模型微调，无需调参。
任务算术为适应基础模型提供了一种模块化、可扩展的方法。然而，组合多个任务向量可能导致跨任务干扰，引发表示漂移并降低性能。表示漂移正则化可自然缓解此问题，但现有方法通常需要外部任务数据，违背了模块化与数据可用性约束（如隐私要求）。本文提出一种无需数据的方法，将正则化对抗表示漂移建模为曲率矩阵近似问题。由此可借助成熟技术，特别采用克罗内克分解近似曲率（Kronecker-Factored Approximate Curvature），获得一种实用的正则项，在任务加法与反向操作中取得当前最优结果。该方法在任务数量上具有恒定复杂度，对任务向量缩放具有鲁棒性，无需保留验证集调参。
原文摘要 · Abstract (English)
Task Arithmetic yields a modular, scalable way to adapt foundation models. Combining multiple task vectors, however, can lead to cross-task interference, causing representation drift and degraded performance. Representation drift regularization provides a natural remedy to disentangle task vectors; however, existing approaches typically require external task data, conflicting with modularity and data availability constraints (e.g., privacy requirements). We propose a dataless approach by framing regularization against representation drift as a curvature matrix approximation problem. This allows us to leverage well-established techniques; in particular, we adopt Kronecker-Factored Approximate Curvature and obtain a practical regularizer that achieves state-of-the-art results in task addition and negation. Our method has constant complexity in the number of tasks and promotes robustness to task vector rescaling, eliminating the need for held-out tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。