插值训练会严重损害回归模型的抗攻击能力。
On damage of interpolation to adversarial robustness in regression
- 在非参数回归框架下研究插值估计器的鲁棒性
- 完美拟合会显著降低模型对微小扰动的抵抗力
- 揭示了高插值率下的'简单规模诅咒'现象
深度神经网络通常具有大量参数，训练至零或接近零的训练误差。尽管存在插值现象，它们在未见数据上仍表现出强大的泛化能力，这引发了广泛的理论研究。已有结果表明，在平方损失下插值未必影响最优收敛速率。然而，深度神经网络对未来的输入扰动极为敏感。一个自然问题是：插值能否避免未来$X$-攻击下的次优性能？本文在非参数回归框架下研究插值估计器的对抗鲁棒性。发现插值估计器即使在轻微的未来$X$-攻击下也必然表现次优，完美拟合会严重损害其鲁棒性。我们还揭示并讨论了一种在高插值率下的有趣现象，称之为'简单规模诅咒'。数值实验支持了理论发现。
原文摘要 · Abstract (English)
Deep neural networks (DNNs) typically involve a large number of parameters and are trained to achieve zero or near-zero training error. Despite such interpolation, they often exhibit strong generalization performance on unseen data, a phenomenon that has motivated extensive theoretical investigations. Comforting results show that interpolation indeed may not affect the minimax rate of convergence under the squared error loss. In the mean time, DNNs are well known to be highly vulnerable to adversarial perturbations in future inputs. A natural question then arises: Can interpolation also escape from suboptimal performance under a future $X$-attack? In this paper, we investigate the adversarial robustness of interpolating estimators in a framework of nonparametric regression. A finding is that interpolating estimators must be suboptimal even under a subtle future $X$-attack, and achieving perfect fitting can substantially damage their robustness. An interesting phenomenon in the high interpolation regime, which we term the curse of simple size, is also revealed and discussed. Numerical experiments support our theoretical findings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。