模型技能合并效果受输入格式影响，同一组合在不同提示下表现差异巨大。
When Do Task Vectors Interfere? Mapping the Validity Boundaries of Weight-Space Composition
- 通过权重空间相加实现多任务合并，但效果依赖输入提示方式。
- 相同模型在代码提示下交互效应达+6.9分，数学提示下为+0.3分。
- 训练时的提示模板对合并性能有决定性影响，适合评估模型融合的实践者必读。
任务算术通过添加权重位移来合并技能，再在基准套件上评估合并模型。我们测量这种组合何时具有功能可加性，发现结果不仅取决于合并的任务，还强烈依赖于模型的提示方式。在五个从0.5B到8B参数量的模型设置、两种方法（LoRA与全量微调）下，成对非可加性真实存在且种子稳定，并可粗略推广至未见任务对：全部八项预先注册的符号预测均成立。然而，这种非可加性在所有测量场景中均呈现输入依赖性：同一合并模型在代码提示下表现出6.9分的交互对比，在数学提示下则无差异；将相同代码提示包裹在适配器训练所用的指令模板中，对比度下降二十倍，从+6.9降至+0.3；而用未经训练的聊天模板重排后，对比度仍保持+12.5，推翻了我们自身的预判。执行基准（pass@1）也继承了训练格式模板的盲区。因此，权重空间组合仅支持粗粒度、输入与格式相关的功能陈述，而非通用的性能预测工具，且训练格式评估无法察觉其真实表现。
原文摘要 · Abstract (English)
Task arithmetic composes skills by adding weight displacements, and merged models are then judged on benchmark suites. We measure when that composition is functionally additive, and find that the answer depends as much on how the model is prompted as on which tasks are merged. Across two-dimensional composition surfaces -- five model settings from 0.5B to 8B, two families, LoRA and full fine-tuning -- pairwise non-additivity is real, seed-stable, and transfers in coarse order to unseen task pairs: all eight preregistered sign predictions held. But it is input-conditioned everywhere we measured: the same merged model that shows a six-point interaction contrast on code prompts shows none on math prompts, and wrapping the identical code prompts in the instruction template the adapters were trained on collapses the contrast twenty-fold, from +6.9 to +0.3 points -- while re-serializing them in an untrained chat template leaves it intact (+12.5), falsifying our own preregistered prediction. Execution benchmarks (pass@1) inherit the training-format wrapper's blindness. Weight-space composition therefore supports coarse, input- and format-conditioned functional statements -- not a universal merging-performance predictor, and not one that training-format evaluations can see.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。