通过动态调整 Lipschitz 常数,让神经网络同时具备高准确率、强鲁棒性和良好校准性。
LiST: Lipschitz Scaling Training for Robust and Calibrated Neural Networks

- 提出 LiST 训练法,自动调节全局 Lipschitz 常数以平衡精度与鲁棒性。
- 在 CIFAR-10/100 和 Tiny-ImageNet 上实现高精度、强鲁棒且无需后处理即校准。
- 支持用户在保持校准的前提下自由选择精度-鲁棒性权衡点,适合实际部署场景。
准确率、鲁棒性和校准性对可靠神经网络至关重要,但常被分别研究;同时满足三者仍是核心挑战。Lipschitz 约束模型可保证鲁棒性,但手动设定约束值 L 会决定准确率-鲁棒性权衡,且其校准性能尚未充分探索。本文揭示了强制 Lipschitz 约束与 Temperature Scaling 之间的理论与实证联系:在给定训练方案下,存在一个非平凡的最优约束值 L*,可直接生成校准网络,且校准可作为在准确率-鲁棒性帕累托前沿上选取理想工作点的合理标准。基于此,我们提出 Lipschitz Scaling Training (LiST),一种迭代调整全局 Lipschitz 常数的新型训练范式。通过训练损失中的边际参数,LiST 还能构建完整校准的帕累托前沿,使用户在维持校准的同时灵活权衡准确率与鲁棒性。收敛后,LiST 可将校准数据重新融入训练,提升样本效率而不牺牲校准性能。我们在 CIFAR-10/100 与 Tiny-ImageNet 上验证了 LiST,结果表明其在对抗攻击和非对抗攻击下均达到与约束及无约束基线相当的精度与鲁棒性,且无需后处理即具备良好校准性。代码已开源。
原文摘要 · Abstract (English)
While accuracy, robustness, and calibration are all essential for reliable neural networks, they are often studied separately; developing models that satisfy all three simultaneously remains a central challenge. Lipschitz-constrained models guarantee robustness by design, yet the manual selection of the Lipschitz constraint L governs the resulting accuracy-robustness trade-off, and their calibration properties remain largely underexplored. In this work, we highlight a theoretical and empirical link between the enforced Lipschitz constraint and Temperature Scaling, a state-of-the-art calibration method. Specifically, we find that for a given training scheme, there exists a non-trivial value L* that yields an out-of-the-box calibrated network, and that calibration acts as a principled criterion to select a well-defined operating point on the accuracy-robustness Pareto front. Leveraging these insights, we introduce Lipschitz Scaling Training (LiST), a novel training paradigm that iteratively adjusts the global Lipschitz constant to reach this operating point. Through a margin parameter in the training loss, LiST further enables the construction of a fully calibrated Pareto front, allowing users to navigate the accuracy-robustness trade-off while remaining calibrated throughout. At convergence, LiST also enables the reintegration of calibration data into training, improving sample efficiency without sacrificing calibration. We validate LiST on CIFAR-10/100 and Tiny-ImageNet, demonstrating competitive accuracy and robustness against constrained and unconstrained baselines, while remaining calibrated out of the box. Code is available at GitHub.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。