提出可随输入变化噪声方差的双随机平滑方法,突破传统全局方差限制。
Dual Randomized Smoothing: Beyond Global Noise Variance
- 引入输入相关噪声方差,通过局部常数约束保证平滑有效性
- 在CIFAR-10上小大半径均优于全局方差方法,最大提升20.0%
- 计算开销仅增加60%,适合实际部署的鲁棒性验证
随机平滑(RS)是认证神经网络对抗扰动鲁棒性的主流方法。标准RS使用全局噪声方差,导致小半径与大半径下难以同时获得高精度:小半径需小方差,大半径需大方差。现有方法受限于全局方差,无法兼顾两者。为此,本文提出双随机平滑(Dual RS)框架,实现输入依赖的噪声方差。理论证明:只要方差在每个输入附近局部恒定,RS仍有效。核心设计包括:(i) 方差估计器为每个输入预测最优噪声方差,(ii) 该估计值由标准RS分类器使用。方差估计器自身也通过RS平滑以确保局部恒定,支持灵活设计。还提出训练策略协同优化两组件。在CIFAR-10上的实验表明,该方法在小半径与大半径下均显著超越全局方差方法,在半径0.5、0.75、1.0分别取得15.6%、20.0%、15.7%的性能提升,推理开销仅增加60%。在ImageNet上,所有半径下均有优势,半径0.5、1.0、1.5分别提升8.6%、17.1%、9.1%。此外,该框架提供路由视角,利用现成专家模型改善准确率-鲁棒性权衡。
原文摘要 · Abstract (English)
Randomized Smoothing (RS) is a prominent technique for certifying the robustness of neural networks against adversarial perturbations. With RS, achieving high accuracy at small radii requires a small noise variance, while achieving high accuracy at large radii requires a large noise variance. However, the global noise variance used in the standard RS formulation leads to a fundamental limitation: there exists no global noise variance that simultaneously achieves strong performance at both small and large radii. To break through the global variance limitation, we propose a dual RS framework which enables input-dependent noise variances. To achieve that, we first prove that RS remains valid with input-dependent noise variances, provided the variance is locally constant around each input. Building on this result, we introduce two components: (i) a variance estimator predicts an optimal noise variance for each input, (ii) this estimated variance is then used by a standard RS classifier. The variance estimator is independently smoothed via RS to ensure local constancy, enabling flexible design. We also introduce training strategies to iteratively optimize the two components. Experiments on CIFAR-10 demonstrate that our dual RS method provides strong performance for both small and large radii-unattainable with global noise variance-while incurring only a 60% computational overhead at inference. Moreover, it outperforms prior input-dependent noise approaches across most radii, with gains at radii 0.5, 0.75, and 1.0 of 15.6%, 20.0%, and 15.7%. On ImageNet, dual RS remains effective across all radii, with advantages of 8.6%, 17.1%, and 9.1% at radii 0.5, 1.0, and 1.5. Additionally, the dual RS framework provides a routing perspective for certified robustness, improving the accuracy-robustness trade-off with off-the-shelf expert RS models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。