提出新型噪声机制GCh,提升模型校准与鲁棒性
Variational Kernel Design for Internal Noise: Gaussian Chaos Noise, Representation Compatibility, and Reliable Deep Learning
- 基于变分核设计框架,构建具几何特性的高斯混沌噪声
- 在ImageNet上实现更优校准,迁移下仍保持低NLL与竞争力准确率
- 适合关注模型可靠性与内在噪声机制的研究者
深度网络中的内部噪声通常依赖于启发式方法,如丢弃、硬掩码或加性扰动。本文提出两个核心问题:内部噪声应具备何种相关性几何结构?其施加的扰动是否与表征相容?通过变分核设计(VKD)框架,将噪声机制定义为分布族、相关核与注入算子,并由学习目标推导得出。在特定空间子类中,对潜在对数场采用二次最大熵原则,得到以Dirichlet-Laplacian为精度的高斯最优解,诱导出Dirichlet格林核几何。经Wick归一化后,得到标准正均值一的门控机制——高斯混沌噪声(GCh)。对于实际使用的样本级门控,证明了其对成对对数比变形的精确高斯控制、边缘敏感的排序稳定性及精确期望内在粗糙度预算;而硬二值掩码则会在正相干表征上引发奇点或相干性增强畸变。在ImageNet与ImageNet-C上,GCh一致提升校准性能,迁移场景下亦改善负对数似然且保持竞争力准确率。
原文摘要 · Abstract (English)
Internal noise in deep networks is usually inherited from heuristics such as dropout, hard masking, or additive perturbation. We ask two questions: what correlation geometry should internal noise have, and is the implemented perturbation compatible with the representations it acts on? We answer these questions through Variational Kernel Design (VKD), a framework in which a noise mechanism is specified by a law family, a correlation kernel, and an injection operator, and is derived from learning desiderata. In a solved spatial subfamily, a quadratic maximum-entropy principle over latent log-fields yields a Gaussian optimizer with precision given by the Dirichlet Laplacian, so the induced geometry is the Dirichlet Green kernel. Wick normalization then gives a canonical positive mean-one gate, Gaussian Chaos Noise (GCh). For the sample-wise gate used in practice, we prove exact Gaussian control of pairwise log-ratio deformation, margin-sensitive ranking stability, and an exact expected intrinsic roughness budget; hard binary masks instead induce singular or coherence-amplified distortions on positive coherent representations. On ImageNet and ImageNet-C, GCh consistently improves calibration and under shift also improves NLL at competitive accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。