模型复杂度越低，对抗鲁棒性越强，可用作新评估指标。
Complexity Matters: Effective Dimensionality as a Measure for Adversarial Robustness
- 用有效维度衡量模型复杂度，揭示其与鲁棒性的反向关系。
- 在YOLO、ResNet等大模型上验证，维度越低鲁棒性越强。
- 适合关注模型安全性的研究人员和工程选型者。
为实现模型选择、对抗训练方法开发及趋势预测的统一评估，亟需一个能量化鲁棒性的单一指标。现有参数量等指标已被证明不足，边界厚度、梯度平坦度等也非可靠代理。本文研究模型有效维度（即复杂度）与对抗鲁棒性的关系。在实际部署的大型模型如YOLO和ResNet上进行实验，发现有效维度与鲁棒性呈近似线性的反向关系：维度越低，鲁棒性越强。进一步分析多种对抗训练方法对有效维度的影响，仍保持该反向线性关系，表明有效维度可作为比参数量或已有指标更精细、有效的鲁棒性评估标准。
原文摘要 · Abstract (English)
Quantifying robustness in a single measure for the purposes of model selection, development of adversarial training methods, and anticipating trends has so far been elusive. The simplest metric to consider is the number of trainable parameters in a model but this has previously been shown to be insufficient at explaining robustness properties. A variety of other metrics, such as ones based on boundary thickness and gradient flatness have been proposed but have been shown to be inadequate proxies for robustness. In this work, we investigate the relationship between a model's effective dimensionality, which can be thought of as model complexity, and its robustness properties. We run experiments on commercial-scale models that are often used in real-world environments such as YOLO and ResNet. We reveal a near-linear inverse relationship between effective dimensionality and adversarial robustness, that is models with a lower dimensionality exhibit better robustness. We investigate the effect of a variety of adversarial training methods on effective dimensionality and find the same inverse linear relationship present, suggesting that effective dimensionality can serve as a useful criterion for model selection and robustness evaluation, providing a more nuanced and effective metric than parameter count or previously-tested measures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。