不训练即可在不同宽度模型间迁移任务更新，提升适配效率。
Transporting Task Vectors across Different Architectures without Training
- 用中间特征的函数效应定义任务向量，实现跨架构迁移。
- 通过正交Procrustes对齐表示空间，闭式求解稳定可靠。
- 无需训练或反向传播，适用于视觉与语言多模型场景。
将大型预训练模型适配到下游任务时，通常会产生代价高昂的任务特定参数更新。尽管已有研究证明相同架构间可迁移此类更新，但不同宽度模型间的迁移仍待探索。本文提出Theseus，一种无需训练的跨异构宽度模型任务更新传输方法。不同于直接匹配参数，我们以任务更新对中间表示产生的函数效应来表征任务向量，并将其形式化为观测激活上的函数匹配问题。经由正交Procrustes分析对齐表示空间后，该问题具有稳定的闭式解，能保持更新的几何结构。我们在视觉与语言模型上验证了Theseus，结果表明其在不同宽度模型间均优于基线方法，且无需额外训练或反向传播。结果表明：当任务身份以函数而非参数方式定义时，任务更新可在不同架构间有意义地迁移。代码见https://github.com/apanariello4/merge-and-rebase。
原文摘要 · Abstract (English)
Adapting large pre-trained models to downstream tasks often produces task-specific parameter updates that are expensive to relearn for every model variant. While recent work has shown that such updates can be transferred between models with identical architectures, transferring them across models of different widths remains unexplored. In this work, we introduce Theseus, a training-free method for transporting task updates across heterogeneous-width models. Rather than matching parameters, we characterize a task update by the functional effect it induces on intermediate representations. We formalize task-vector transport as a functional matching problem on observed activations and show that, after aligning representation spaces via orthogonal Procrustes analysis, it admits a stable closed-form solution that preserves the geometry of the update. We evaluate Theseus on vision and language models across different widths, showing consistent improvements over baselines without additional training or backpropagation. Our results show that task updates can be meaningfully transferred across architectures when task identity is defined functionally rather than parametrically. Code is available at https://github.com/apanariello4/merge-and-rebase.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。