arXiv:2501.10466cs.LGcs.AI2025-01中稿 · ICML

通过聚类筛选关键样本,让对抗训练更省时省力。

Efficient Semi-Supervised Adversarial Training via Latent Clustering-Based Data Reduction

  • 用隐空间聚类挑选靠近决策边界的少数关键样本
  • 仅需原数据量1/10,就能达到相近的抗攻击准确率
  • 适合追求高效鲁棒模型的工程与研究者

在对抗环境下训练鲁棒模型通常需要大量样本。近期的半监督对抗训练(SSAT)虽为当前最优方法,但依赖大量额外数据,导致训练时间长、内存消耗高。本文提出基于隐空间聚类的数据精简策略,通过k-means等方法选择位于模型决策边界附近的少量关键样本,同时保持边界与非边界样本的平衡比例,避免过拟合。在多个图像基准测试中,所提方法显著降低对额外数据的需求和计算成本,仍保持强鲁棒性。其中,基于k-means的隐空间选取与结合LCG-KM的引导扩散方法表现最佳,使用5至10倍更少的无标签数据即可实现接近全量训练的鲁棒准确率;相比完全收敛的SSAT,总运行时间减少约3至4倍。

原文摘要 · Abstract (English)

Learning robust models under adversarial settings is widely recognized as requiring a considerably large number of training samples. Recent work proposes semi-supervised adversarial training (SSAT), which utilizes external unlabeled or synthetically generated data and is currently the state of the art. However, SSAT requires substantial extra data to attain high robustness, resulting in prolonged training time and increased memory usage. In this paper, we propose data reduction strategies to improve the efficiency of SSAT by optimizing the amount of additional data incorporated. Specifically, we design novel latent clustering-based techniques to select or generate a small, critical subset of data samples near the model's decision boundary. While focusing on boundary-adjacent points, our methods maintain a balanced ratio between boundary and non-boundary data points, thereby avoiding overfitting. Comprehensive experiments across image benchmarks demonstrate that our methods can effectively reduce SSAT's data requirements and computational costs while preserving its strong robustness advantages. In particular, our latent-space selection scheme based on k-means clustering and our guided diffusion-based approach with LCG-KM are the most effective, achieving nearly identical robust accuracies with 5 times to 10 times less unlabeled data. When compared to full SSAT trained to convergence, our methods reduce total runtime by approximately 3 times to 4 times due to strategic prioritization of unlabeled data.

对抗训练数据精简聚类高效学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。