根据梯度波动性动态注入噪声，提升模型泛化能力
VISP: Volatility Informed Stochastic Projection for Adaptive Regularization
- 用梯度波动性决定噪声强度，动态调节正则化力度
- 在多个数据集上均优于固定噪声和基线模型
- 适合需要稳定训练过程的深度学习应用
我们提出VISP：一种基于梯度波动性的自适应正则化方法。与传统均匀加噪或固定丢弃率不同，VISP通过梯度统计量动态计算波动性，并据此缩放随机投影矩阵。该机制对梯度波动大的输入和隐藏节点进行选择性正则化，同时保留稳定表征，有效缓解过拟合。在MNIST、CIFAR-10和SVHN上的大量实验表明，VISP在泛化性能上持续优于基线模型和固定噪声方法。对波动性演化、投影矩阵谱特性及激活分布的分析显示，VISP不仅稳定了网络内部动态，还促进了更鲁棒的特征表示。
原文摘要 · Abstract (English)
We propose VISP: Volatility Informed Stochastic Projection, an adaptive regularization method that leverages gradient volatility to guide stochastic noise injection in deep neural networks. Unlike conventional techniques that apply uniform noise or fixed dropout rates, VISP dynamically computes volatility from gradient statistics and uses it to scale a stochastic projection matrix. This mechanism selectively regularizes inputs and hidden nodes that exhibit higher gradient volatility while preserving stable representations, thereby mitigating overfitting. Extensive experiments on MNIST, CIFAR-10, and SVHN demonstrate that VISP consistently improves generalization performance over baseline models and fixed-noise alternatives. In addition, detailed analyses of the evolution of volatility, the spectral properties of the projection matrix, and activation distributions reveal that VISP not only stabilizes the internal dynamics of the network but also fosters a more robust feature representation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。