无需预训练模型，在极低标签率下实现强对抗鲁棒性。
Revisiting Semi-supervised Adversarial Robustness via Noise-aware Online Robust Distillation
- 用噪声感知的在线鲁棒蒸馏增强伪标签，提升半监督对抗训练效果。
- 在0.1%~10%标签下，对抗攻击准确率相对提升90%（ε=8/255）。
- 框架简洁可扩展，适合资源受限场景或与现有预训练策略结合使用。
鲁棒自训练（RST）框架已成为半监督对抗训练的重要方法。为探索在更低标注预算下处理更复杂任务的可能性，本文提出SNORD——一种无需依赖鲁棒预训练模型的简单而有效框架，将当代半监督学习技术引入对抗训练领域。通过改进伪标签并更有效地管理噪声数据，SNORD在多种数据集和标注比例下均展现出卓越且领先的性能。相比全对抗监督，其在CIFAR-10、CIFAR-100和TinyImageNet-200上分别仅需小于0.1%、2%和10%的标签，即可在ε=8/255的AutoAttack下达到90%的相对鲁棒准确率。额外实验验证了各组件的有效性，并展示了与现有对抗预训练策略结合以进一步提升鲁棒性的可行性。
原文摘要 · Abstract (English)
The robust self-training (RST) framework has emerged as a prominent approach for semi-supervised adversarial training. To explore the possibility of tackling more complicated tasks with even lower labeling budgets, unlike prior approaches that rely on robust pretrained models, we present SNORD - a simple yet effective framework that introduces contemporary semi-supervised learning techniques into the realm of adversarial training. By enhancing pseudo labels and managing noisy training data more effectively, SNORD showcases impressive, state-of-the-art performance across diverse datasets and labeling budgets, all without the need for pretrained models. Compared to full adversarial supervision, SNORD achieves a 90% relative robust accuracy under epsilon = 8/255 AutoAttack, requiring less than 0.1%, 2%, and 10% labels for CIFAR-10, CIFAR-100, and TinyImageNet-200, respectively. Additional experiments confirm the efficacy of each component and demonstrate the adaptability of integrating SNORD with existing adversarial pretraining strategies to further bolster robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。