提出首个针对数据蒸馏的对抗鲁棒性评测基准,解决安全评估缺失问题。
BEARD: Benchmarking the Adversarial Robustness for Dataset Distillation
- 构建统一评测框架,涵盖多种攻击与蒸馏方法
- 在CIFAR-10/100等数据集上验证不同压缩比下的鲁棒性差异
- 适合关注模型安全与可复现研究的研究者
数据蒸馏(DD)是一种新兴技术,能将大规模数据集压缩为更小的合成数据集,同时保持高测试性能并支持大模型高效训练。然而,当前研究主要聚焦于有限压缩比下的准确率提升,常忽视对抗鲁棒性等关键安全问题。评估该鲁棒性的核心挑战在于蒸馏方法、模型架构与攻击策略之间的复杂交互,导致标准化评估困难。为此,我们提出BEARD——一个开放、统一的基准,用于系统评估包括DM、IDM和BACON在内的DD方法的对抗鲁棒性。BEARD覆盖了多种对抗攻击(如FGSM、PGD、C&W)及蒸馏数据集(如CIFAR-10/100、TinyImageNet)。通过对抗博弈框架,引入三个关键指标:鲁棒性比率(RR)、攻击效率比率(AE)和综合鲁棒性-效率指数(CREI)。分析包含统一基准、不同每类图像数(IPC)设置以及对抗训练的影响。结果公开于BEARD排行榜,并提供模型与数据集库以支持可复现研究。代码可访问BEARD。
原文摘要 · Abstract (English)
Dataset Distillation (DD) is an emerging technique that compresses large-scale datasets into significantly smaller synthesized datasets while preserving high test performance and enabling the efficient training of large models. However, current research primarily focuses on enhancing evaluation accuracy under limited compression ratios, often overlooking critical security concerns such as adversarial robustness. A key challenge in evaluating this robustness lies in the complex interactions between distillation methods, model architectures, and adversarial attack strategies, which complicate standardized assessments. To address this, we introduce BEARD, an open and unified benchmark designed to systematically assess the adversarial robustness of DD methods, including DM, IDM, and BACON. BEARD encompasses a variety of adversarial attacks (e.g., FGSM, PGD, C&W) on distilled datasets like CIFAR-10/100 and TinyImageNet. Utilizing an adversarial game framework, it introduces three key metrics: Robustness Ratio (RR), Attack Efficiency Ratio (AE), and Comprehensive Robustness-Efficiency Index (CREI). Our analysis includes unified benchmarks, various Images Per Class (IPC) settings, and the effects of adversarial training. Results are available on the BEARD Leaderboard, along with a library providing model and dataset pools to support reproducible research. Access the code at BEARD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。