图像分类模型的判别依赖相位而非幅度，揭示了深层表征中的隐含规律。
The Importance of Phase in Neural Representations: An Internal Oppenheim-Lim Test of Image Classifiers
- 通过相位移植实验，发现模型预测由输入图像的相位主导。
- 移除特定幅度后模型准确率几乎不变，证明幅度对识别作用有限。
- 不同架构暴露相位信息的方式不同，解释了卷积与注意力模型的差异。
Oppenheim和Lim（1981）发现，仅用傅里叶相位重建自然图像仍可辨识，而幅度对身份信息贡献甚微。本文探究训练后的图像分类器是否在隐藏层中复现这一不对称性，并进行因果干预：将两个图像中某一层次的相位移植到另一图像的幅度上，观察预测结果跟随哪个来源。在PRISM2D、GFNet和ViT-B/16中，预测始终跟随相位或符号来源，即使删除所有图像特异性幅度，准确率也基本不变，表明身份信息主要由相位承载。ResNet-50起初看似不符，但其ReLU前的干预显示晚期块存在强符号编码；直流项控制实验进一步表明读出机制依赖通道级空间均值。各类控制排除了幅度完全失敏的可能。因此，不同架构共享相位/符号身份编码，但在不同基底下显现，取决于整流与读出结构，为卷积神经网络与注意力模型之间的纹理-形状差距提供了机制解释。
原文摘要 · Abstract (English)
Oppenheim and Lim (1981) showed that natural images stay recognizable when reconstructed from their Fourier phase alone, while the magnitude carries little of their identity. We ask whether trained image classifiers reproduce this asymmetry inside their hidden layers, and we test it causally: given two images, we transplant the phase of one onto the magnitude of the other at a chosen layer and record which image the prediction follows. In PRISM2D, GFNet, and ViT-B/16 the prediction follows the phase or sign donor, and deleting all image-specific magnitude barely moves accuracy, so identity rides on phase while image-specific magnitude is largely dispensable to the readout. ResNet-50 at first seems to break the pattern, because transplanting sign after its ReLUs does nothing; a fair intervention before the ReLU reveals a strong latent sign code in the late blocks, and a DC-only control shows the readout consumes a channel-wise spatial average. Controls rule out the trivial case in which magnitude simply stops depending on the image. The architectures therefore share a phase/sign identity code but expose it in different bases, set by rectification and readout geometry, which gives a mechanistic account of the texture--shape gap between CNNs and attention models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。