用几何方法揭示人与模型在视觉表征上的深层差异。
Geometry matters: insights from Ollivier Ricci Curvature and Ricci Flow into representational alignment through Ollivier-Ricci Curvature and Ricci Flow
- 引入奥利维耶里奇曲率和里奇流分析表征的局部几何结构。
- 发现2D到3D视角转变时模型与人类表征存在几何不一致。
- 适合研究认知科学与神经网络对齐的学者参考。
表征相似性分析（RSA）常用于比较人类与神经网络的表征对齐，但其结论可能因忽略底层表示几何而产生误导。本文提出一种基于奥利维耶里奇曲率和里奇流的框架，用于分析表征的细粒度局部几何结构。该方法不依赖表征空间来源，可直接比较人类行为判断与模型向量嵌入的几何一致性。我们将其应用于2D与3D人脸刺激，对比了基础2D网络（VGG-Face）及其与人类行为对齐的变体。结果表明，几何感知分析能更敏感地揭示表征间的差异与几何不匹配，尤其在从2D到3D视图转换时暴露了显著的几何不一致。这说明融入几何信息可发现传统度量未能捕捉的对齐差异，深化对表征组织的理解。
原文摘要 · Abstract (English)
Representational similarity analysis (RSA) is widely used to analyze the alignment between humans and neural networks; however, conclusions based on this approach can be misleading without considering the underlying representational geometry. Our work introduces a framework using Ollivier Ricci Curvature and Ricci Flow to analyze the fine-grained local structure of representations. This approach is agnostic to the source of the representational space, enabling a direct geometric comparison between human behavioral judgments and a model's vector embeddings. We apply it to compare human similarity judgments for 2D and 3D face stimuli with a baseline 2D native network (VGG-Face) and a variant of it aligned to human behavior. Our results suggest that geometry-aware analysis provides a more sensitive characterization of discrepancies and geometric dissimilarities in the underlying representations that remain only partially captured by RSA. Notably, we reveal geometric inconsistencies in the alignment when moving from 2D to 3D viewing conditions.This highlights how incorporating geometric information can expose alignment differences missed by traditional metrics, offering deeper insight into representational organization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。