用曲线连接不同抗扰模型,高效提升多范数对抗防御能力。
Robust CurveMoE: Multi-Norm Adversarial Defense for Mixture-of-Experts Models via Mode Connectivity

- 通过低损耗路径连接针对不同扰动范数的专家模型。
- 在CIFAR-100和ImageNet-100上联合准确率分别提升2.37和2.13个百分点。
- 仅优化关键层参数,大幅降低训练成本,适合资源受限场景。
多范数对抗防御旨在保护神经网络免受不同范数约束扰动的影响,但现有方法通常在单一参数配置下优化相互竞争的鲁棒性目标,导致训练成本高且鲁棒性存在不利权衡。我们提出Robust CurveMoE,一种高效的专家混合框架,通过低损失路径连接针对不同扰动范数的专用模型,并利用该路径上模型的互补鲁棒性特征。Robust CurveMoE从鲁棒性约束的曲线位置中提取干净样本与范数专用专家,仅选择性地专家化有影响力的层,其余参数在路由路径间共享。为进一步降低曲线构建成本,引入基于贡献的局部更新机制,使用初始化梯度得分筛选关键曲线参数。我们还理论上界定了部分与完整曲线优化之间的目标差距。在WideResNet和Vision Transformer架构下对CIFAR-100与ImageNet-100的实验表明,Robust CurveMoE在干净准确率、范数特定准确率及联合准确率上均优于MSD与ERMC。尤其在联合准确率上,相比最强基线分别提升2.37和2.13个百分点。大量消融实验进一步验证了局部更新、选择性专家化及鲁棒性约束专家选择的有效性。
原文摘要 · Abstract (English)
Multi-norm adversarial defense aims to protect neural networks against perturbations defined by different norm constraints, but existing methods typically optimize competing robustness objectives within a single parameter configuration, leading to substantial training cost and unfavorable robustness trade-offs. We propose Robust CurveMoE, an efficient mixture-of-experts framework that connects models specialized for different perturbation norms through a low-loss path and exploits the complementary robustness profiles of models along this path. Robust CurveMoE derives clean and norm-specialized experts from robustness-constrained curve locations and selectively expertizes only influential layers, while sharing the remaining parameters across routing paths. To further reduce curve-construction cost, we introduce contribution-guided partial updating, which selects influential curve parameters using initialization-based gradient scores. We also theoretically bound the objective gap between partial and full curve optimization. Experiments on CIFAR-100 and ImageNet-100 with WideResNet and Vision Transformer architectures show that Robust CurveMoE consistently improves clean, norm-specific, and Union accuracy over MSD and ERMC. In particular, it improves Union accuracy by 2.37 and 2.13 percentage points over the strongest baseline on CIFAR-100 and ImageNet-100, respectively. Extensive ablations further validate the effectiveness of partial updating, selective expertization, and robustness-constrained expert selection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。