不同数据集训练的CNN模型，其特征编码具有普遍性。
On the universality of neural encodings in CNNs
- 通过分解空间与通道维度，直接比较权重相似性
- VGG类网络多层的特征向量在不同数据集上高度一致
- 为迁移学习提供理论基础，适合研究模型泛化者
我们研究了在图像分类任务中训练的卷积神经网络（CNN）中神经编码的普遍性。提出一种直接比较学习权重的方法，基于空间与通道维度的分解，测量对齐权重协方差的相似性。结果显示，在VGG型网络的多个层级中，学习到的特征向量在不同自然图像数据集间表现出显著一致性。结果表明，自然图像存在一种普遍的神经编码机制，从更根本层面解释了迁移学习的成功。该工作提示，构建基础模型时，可优先追求编码的普遍性，而非单纯提升性能。
原文摘要 · Abstract (English)
We explore the universality of neural encodings in convolutional neural networks trained on image classification tasks. We develop a procedure to directly compare the learned weights rather than their representations. It is based on a factorization of spatial and channel dimensions and measures the similarity of aligned weight covariances. We show that, for a range of layers of VGG-type networks, the learned eigenvectors appear to be universal across different natural image datasets. Our results suggest the existence of a universal neural encoding for natural images. They explain, at a more fundamental level, the success of transfer learning. Our work shows that, instead of aiming at maximizing the performance of neural networks, one can alternatively attempt to maximize the universality of the learned encoding, in order to build a principled foundation model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。