随机初始化的空间混合也能达到高精度，还更抗对抗攻击。
Rethinking the Role of Spatial Mixing
- 固定随机空间混合作用，不训练也能保持性能
- 在ResNet和ConvMixer上表现接近全训练模型
- 适合关注模型鲁棒性与机制理解的研究者
长期以来，2D卷积是主流视觉模型的核心，其同时混合特征图的时空与通道信息。近年架构多采用解耦空间与通道混合作用的模块化设计，便于分析各自影响。本文发现，在经典（ResNet）与前沿（ConvMixer）模型中，将空间混合作用保持在随机初始化状态，仍可达到几乎相同的分类性能。进一步表明，此类固定随机空间混合模型天然更抗对抗扰动。该现象不仅限于分类任务，还可用于恢复像素打乱图像，揭示空间混合在表征学习中的潜在作用。
原文摘要 · Abstract (English)
Until quite recently, the backbone of nearly every state-of-the-art computer vision model has been the 2D convolution. At its core, a 2D convolution simultaneously mixes information across both the spatial and channel dimensions of a representation. Many recent computer vision architectures consist of sequences of isotropic blocks that disentangle the spatial and channel-mixing components. This separation of the operations allows us to more closely juxtapose the effects of spatial and channel mixing in deep learning. In this paper, we take an initial step towards garnering a deeper understanding of the roles of these mixing operations. Through our experiments and analysis, we discover that on both classical (ResNet) and cutting-edge (ConvMixer) models, we can reach nearly the same level of classification performance by and leaving the spatial mixers at their random initializations. Furthermore, we show that models with random, fixed spatial mixing are naturally more robust to adversarial perturbations. Lastly, we show that this phenomenon extends past the classification regime, as such models can also decode pixel-shuffled images.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。