用多专家架构和随机对抗训练提升模型抗干扰能力
Divided We Fall: Defending Against Adversarial Attacks via Soft-Gated Fractional Mixture-of-Experts with Randomized Adversarial Training
- 采用九个预训练专家组成的混合专家结构，动态加权选择
- 在CIFAR-10和SVHN上对FGSM和PGD攻击防御效果优于现有方法
- 适合需要高鲁棒性的安全敏感场景，如医疗或自动驾驶
机器学习虽能自动化众多任务，但对对抗性攻击仍显脆弱。对抗攻击通过人眼难以察觉的微小扰动导致模型误判。本文提出一种基于混合专家架构的防御系统，在其中引入对抗训练模块以增强对白盒逃逸攻击的鲁棒性。系统使用九个以ResNet-18为骨干的预训练分类器（专家），在端到端训练中联合优化所有专家与门控机制参数。实验表明，在CIFAR-10和SVHN数据集上，该系统对强白盒攻击（如FGSM和PGD）的防御性能优于现有基于MoE的防御方法。虽然多专家结构增加了训练时间与计算开销，但推理成本随专家数量近似线性增长，远低于训练成本。
原文摘要 · Abstract (English)
Machine learning is a powerful tool enabling full automation of a huge number of tasks without explicit programming. Despite recent progress of machine learning in different domains, these models have shown vulnerabilities when they are exposed to adversarial threats. Adversarial threats aim to hinder the machine learning models from satisfying their objectives. They can create adversarial perturbations, which are imperceptible to humans' eyes but have the ability to cause misclassification during inference. In this paper, we propose a defense system, which devises an adversarial training module within mixture-of-experts architecture to enhance its robustness against white-box evasion attacks. In our proposed defense system, we use nine pre-trained classifiers (experts) with ResNet-18 as their backbone. During end-to-end training, the parameters of all experts and the gating mechanism are jointly updated allowing further optimization of the experts. Our proposed defense system outperforms prior MoE-based defenses under strong white-box FGSM and PGD evaluation on CIFAR-10 and SVHN. The use of multiple experts increases training time and compute relative to single-network baselines; however, inference scales approximately linearly with the number of experts and is substantially cheaper than training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。