arXiv:2606.12498cs.CRcs.LG2026-06被引 1

通过特征空间优化,提升模型合并的抗后门攻击能力。

From Parameters to Feature Space: Task Arithmetic for Backdoor Mitigation in Model Merging

论文配图:From Parameters to Feature Space: Task Arithmetic for Backdoor Mitigation in Model Merging
图 1 · 摘自论文原文
  • 从特征空间出发设计抗后门任务向量,避免直接修改参数。
  • 在全微调和高效微调下均有效抵御后门攻击。
  • 适合关注模型安全与融合性能平衡的研究者。

模型合并(MM)作为一种低成本整合多任务模型的方法受到广泛关注。然而,近期研究发现,模型合并极易遭受后门攻击。现有基于任务算术的防御方法常因依赖参数空间直接编辑,导致清除后门时严重损害正常任务性能。为此,本文提出线性特征路径最小化(LFPM)框架,通过在被污染的合并模型中引入抗后门任务向量,从统一的特征空间视角出发,在跨任务线性(CTL)框架下实现鲁棒性优化。该框架利用任务间特征近似线性的特性,指导抗后门向量的优化,以抑制后门同时保持良好性能。此外,基于梯度累积与损失路径积分的优化机制,确保沿插值路径的强鲁棒性。大量实验表明,LFPM在全微调与参数高效微调(PEFT)设置下均表现出对后门攻击的持续强鲁棒性。

原文摘要 · Abstract (English)

Model merging (MM) has gained significant attention as a cost-effective approach to integrate multiple task-specific models into a unified model. However, recent work reveals that MM is highly susceptible to backdoor attacks. Existing defenses based on task arithmetic often fail to eliminate backdoors without substantially degrading clean-task performance, owing to their reliance on direct parameter-space editing. To address this gap, we propose Linear Feature Path Minimization (LFPM), a backdoor mitigation framework for model merging, which introduces an anti-backdoor task vector into the backdoored merged model. Unlike prior approaches, LFPM formulates the backdoor robustness of the merged model from a unified feature-space perspective under the Cross-Task Linearity (CTL) framework, which leverages the approximate linearity of features across tasks. This perspective guides the optimization of the anti-backdoor task to suppress backdoors while preserving clean-task performance. Furthermore, we introduce an effective optimization mechanism based on gradient accumulation and loss path-integral, ensuring robust backdoor suppression along the interpolation path. Extensive experiments demonstrate that LFPM consistently exhibits strong robustness against backdoor attacks in both full fine-tuning and Parameter-Efficient Fine-Tuning (PEFT) settings.

模型合并后门防御特征空间安全融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。