揭秘视觉模型抗噪能力的架构秘密,给出可直接应用的设计准则。
Unlocking Noise-Resistant Vision: Key Architectural Secrets for Robust Models
- 发现四种提升抗高斯噪声能力的架构模式:大卷积核、小输入分辨率、平均池化、监督式ViT
- 对比实验显示最高提升506名次、准确率提高21.6个百分点
- 理论证明各设计项的降噪机制,适合模型设计者快速优化鲁棒性
尽管视觉模型的鲁棒性常被评估,但其对特定架构设计的选择依赖性却很少被深入剖析。本文通过在1,174个预训练视觉模型上进行广泛评估,实证识别出四种提升高斯噪声鲁棒性的稳定设计模式:更大的茎部卷积核、更小的输入分辨率、平均池化,以及监督式视觉变换器(ViT)而非CLIP ViT,分别带来高达506名次提升和21.6%的准确率增益。随后,我们构建理论分析,将观察到的相关性转化为因果机制:首先证明低通茎部卷积核的噪声衰减程度随卷积核尺寸平方递减;其次表明抗混叠下采样可使噪声能量大致按下采样因子的平方降低;第三,揭示平均池化无偏且噪声抑制与池化窗口面积成正比,而最大池化存在缓慢增长的正偏差,导致均方误差更高、最坏情况敏感性更强;最后,通过像素空间利普希茨界解释了CLIP ViT的脆弱性:其较小的归一化标准差使最坏情况敏感性相比传统Inception风格预处理最高放大1.91倍。结果将鲁棒性拆解为可解释模块,建立解释性理论,并提供即插即用的实用设计指南。
原文摘要 · Abstract (English)
While the robustness of vision models is often measured, their dependence on specific architectural design choices is rarely dissected. We investigate why certain vision architectures are inherently more robust to additive Gaussian noise and convert these empirical insights into simple, actionable design rules. Specifically, we performed extensive evaluations on 1,174 pretrained vision models, empirically identifying four consistent design patterns for improved robustness against Gaussian noise: larger stem kernels, smaller input resolutions, average pooling, and supervised vision transformers (ViTs) rather than CLIP ViTs, which yield up to 506 rank improvements and 21.6\%p accuracy gains. We then develop a theoretical analysis that explains these findings, converting observed correlations into causal mechanisms. First, we prove that low-pass stem kernels attenuate noise with a gain that decreases quadratically with kernel size and that anti-aliased downsampling reduces noise energy roughly in proportion to the square of the downsampling factor. Second, we demonstrate that average pooling is unbiased and suppresses noise in proportion to the pooling window area, whereas max pooling incurs a positive bias that grows slowly with window size and yields a relatively higher mean-squared error and greater worst-case sensitivity. Third, we reveal and explain the vulnerability of CLIP ViTs via a pixel-space Lipschitz bound: The smaller normalization standard deviations used in CLIP preprocessing amplify worst-case sensitivity by up to 1.91 times relative to the Inception-style preprocessing common in supervised ViTs. Our results collectively disentangle robustness into interpretable modules, provide a theory that explains the observed trends, and build practical, plug-and-play guidelines for designing vision models more robust against Gaussian noise.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。