跨模型迁移任务向量，仅用少量样本即可高效更新模型。
Cross-Model Transfer of Task Vectors via Few-Shot Orthogonal Alignment
- 通过少量样本对齐任务向量与目标模型参数空间。
- 在8个分类数据集上表现优于直接应用，接近微调效果。
- 保持任务向量可复用性，适合快速适配不同预训练模型。
任务算术通过参数空间中的向量表示特定任务的修改，实现高效模型编辑。传统方法假设源模型与目标模型共享相同的预训练初始化，限制了其在跨模型迁移场景下的应用，即模型在不同数据集上独立预训练时。为此，我们提出一种基于少样本正交对齐的方法，将任务向量映射到不同预训练的目标模型参数空间中。该变换保留任务向量的范数与秩等关键属性，且仅需少量标注样本即可学习。我们在两个分别在YFCC100M和LAION400M上预训练的视觉变换器（Vision Transformers）上进行评估，并在八个分类数据集上测试。实验结果表明，本方法在迁移准确率上优于直接应用任务向量，性能接近少样本微调，同时保持任务向量的模块化与可重用性。代码已公开于https://github.com/kawakera-lab/CrossModelTransfer。
原文摘要 · Abstract (English)
Task arithmetic enables efficient model editing by representing task-specific changes as vectors in parameter space. Task arithmetic typically assumes that the source and target models are initialized from the same pre-trained parameters. This assumption limits its applicability in cross-model transfer settings, where models are independently pre-trained on different datasets. To address this challenge, we propose a method based on few-shot orthogonal alignment, which aligns task vectors to the parameter space of a differently pre-trained target model. These transformations preserve key properties of task vectors, such as norm and rank, and are learned using only a small number of labeled examples. We evaluate the method using two Vision Transformers pre-trained on YFCC100M and LAION400M, and test on eight classification datasets. Experimental results show that our method improves transfer accuracy over direct task vector application and achieves performance comparable to few-shot fine-tuning, while maintaining the modularity and reusability of task vectors. Our code is available at https://github.com/kawakera-lab/CrossModelTransfer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。