自适应调整噪声方差，显著降低梯度解释中的噪声。
AdaptGrad: Adaptive Sampling to Reduce Noise
- 将平滑梯度视为卷积，从置信度角度重新理解噪声与方差关系。
- 在多个数据集上，相比基线方法，几乎完全消除原始梯度噪声。
- 方法简单通用，可直接提升各类基于梯度的可解释性可视化效果。
梯度平滑是一种有效降低基于梯度模型解释方法中噪声的手段。SmoothGrad通过添加高斯噪声来缓解梯度噪声，但其关键超参数——噪声方差σ——通常手动设定或使用启发式方法，导致平滑后的梯度仍残留一定噪声。本文将SmoothGrad重新理解为卷积的推论，从置信度视角重新审视梯度噪声及σ的作用，并据此提出自适应梯度平滑方法AdaptGrad。大量定性和定量实验表明，AdaptGrad在多个基准测试中显著优于基线方法，几乎完全消除原始梯度噪声。该方法简单且通用，可广泛用于提升基于梯度的可解释性方法的可视化质量。
原文摘要 · Abstract (English)
Gradient Smoothing is an efficient approach to reducing noise in gradient-based model explanation method. SmoothGrad adds Gaussian noise to mitigate much of these noise. However, the crucial hyper-parameter in this method, the variance $σ$ of Gaussian noise, is set manually or with heuristic approach. However, it results in the smoothed gradients still containing a certain amount of noise. In this paper, we aim to interpret SmoothGrad as a corollary of convolution, thereby re-understanding the gradient noise and the role of $σ$ from the perspective of confidence level. Furthermore, we propose an adaptive gradient smoothing method, AdaptGrad, based on these insights. Through comprehensive experiments, both qualitative and quantitative results demonstrate that AdaptGrad could effectively reduce almost all the noise in vanilla gradients compared with baselines methods. AdaptGrad is simple and universal, making it applicable for enhancing gradient-based interpretability methods for better visualization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。