用关系知识蒸馏让神经网络表示更接近人类,实现无监督细粒度对齐。
Relational Knowledge Distillation Brings DNN Representations Close Enough to Humans to Be Aligned Without Supervision

- 通过关系知识蒸馏迁移人类认知结构到神经网络。
- 在独立测试集上实现个体物体级别的无监督对齐。
- 提升的是整体结构相似性,而非局部邻近关系。
将深度神经网络(DNN)的内部表征与人类心理表征关联,对构建人类视觉的计算模型至关重要。现有DNN表征与人类表征仍不充分相似,后者通常通过大规模对象图像相似性判断来测量。一种自然方法是直接将人类表征的关系结构转移至DNN,已有研究显示可提升人-机表征相似性。但该效果在更严格的评估下仍未验证:一是个体物体层面的细粒度对齐,二是泛化到与训练数据无关的人类嵌入。本文采用无监督比较方法——格罗莫夫-瓦瑟斯坦最优传输(GWOT),仅基于内部距离结构估计人-机对应关系,以检验细粒度对齐。进一步在与训练数据无重叠的精选概念测试集上评估泛化能力。结果表明,使用已知的关系转移方法——关系知识蒸馏(RKD)微调预训练DNN,使其在个体物体层面达到足够接近人类的表征,从而实现无监督对齐。该提升源于更接近人类的全局结构(如粗类别间距离排序),而局部最近邻重合率基本不变。这说明从人类迁移关系结构可使预训练DNN的全局结构足够接近人类,进而实现无需监督的细粒度对齐。
原文摘要 · Abstract (English)
Linking the internal representations of deep neural networks (DNNs) to human mental representations is important for using DNNs as computational models of human vision. Existing DNN representations remain insufficiently similar to human mental representations, which are not directly observable and are therefore commonly measured through large-scale similarity judgments of object images. A natural approach to narrowing this gap is to directly transfer the relational structure of human representations into DNNs, and previous studies have reported improved human-DNN representational similarity. However, whether this improvement holds under stricter evaluation remains untested in two respects: fine-grained alignment at the individual-object level, and generalization to a human embedding derived from a dataset independent of the training data. Here, we employ an unsupervised comparison method, Gromov-Wasserstein optimal transport (GWOT), which estimates human-DNN correspondences from the internal distance structure alone and thereby tests fine-grained alignment. We further assess generalization on a curated test set of concepts non-overlapping with the training data. We show that fine-tuning pre-trained DNNs with Relational Knowledge Distillation (RKD), an established relational transfer method, brings DNNs close enough to humans to be aligned at the individual-object level on this test set. We also show that this improvement is driven by a more human-like global structure, as reflected in the ordering of distances among coarse categories, while the local human-DNN nearest-neighbor overlap rate remains largely unchanged. These findings indicate that relational transfer from humans brings the global structure of pre-trained DNNs close enough to the human structure to enable fine-grained human-DNN alignment without supervision.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。