在数据流形上注入几何感知噪声，提升模型泛化与鲁棒性。
Staying on the Manifold: Geometry-Aware Noise Injection
- 将高斯噪声投影到流形切空间，并沿测地线映射回流形
- 在高曲率流形上显著提升泛化性能，降低对超参数敏感度
- 适用于生成模型逼近的流形，适合流形学习与稳健训练场景
已有研究表明，训练时扰动输入可隐式正则化学习函数的梯度，使模型更平滑并增强泛化能力。然而，以往研究多在输入空间中添加环境噪声，未考虑数据的潜在结构。本文提出多种考虑数据低维流形结构的几何感知噪声注入策略：首先将环境高斯噪声投影到流形的切空间，再通过对应测地线曲线映射回流形；还引入沿流形随机移动的布朗运动噪声。实验表明，几何感知噪声在高曲率流形上能显著提升泛化能力和对超参数选择的鲁棒性，而在简单流形上表现不逊于无噪声训练。该框架可扩展至由生成模型近似的流形，在MNIST数字数据集上也观察到相似趋势。
原文摘要 · Abstract (English)
It has been shown that perturbing the input during training implicitly regularises the gradient of the learnt function, leading to smoother models and enhancing generalisation. However, previous research mostly considered the addition of ambient noise in the input space, without considering the underlying structure of the data. In this work, we propose several strategies of adding geometry-aware input noise that accounts for the lower dimensional manifold the input space inhabits. We start by projecting ambient Gaussian noise onto the tangent space of the manifold. In a second step, the noise sample is mapped on the manifold via the associated geodesic curve. We also consider Brownian motion noise, which moves in random steps along the manifold. We show that geometry-aware noise leads to improved generalisation and robustness to hyperparameter selection on highly curved manifolds, while performing at least as well as training without noise on simpler manifolds. Our proposed framework extends to data manifolds approximated by generative models and we observe similar trends on the MNIST digits dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。