提出新方法，从内在几何角度分析神经网络表示差异。
Geometry-aware similarity metrics for neural representations on Riemannian and statistical manifolds
- 基于黎曼几何，比较神经表示的内在几何结构。
- 可区分不同学习策略下的特征计算机制。
- 适用于深度网络、非线性动态与扩散模型分析。
相似性度量广泛用于解释神经网络解决任务时所采用的表征几何。然而，现有方法仅比较状态空间中表示的外在几何，难以捕捉根本不同的神经网络解决方案之间的细微但关键差异。本文提出度量相似性分析（MSA），利用黎曼几何工具，在流形假设下比较神经表示的内在几何。结果表明，MSA可用于：i）分离不同学习策略下深层网络中的神经计算特征；ii）比较非线性动态；iii）研究扩散模型。因此，我们构建了一个数学严谨且普适性强的框架，通过比较内在几何来理解神经计算背后的机制。
原文摘要 · Abstract (English)
Similarity measures are widely used to interpret the representational geometries used by neural networks to solve tasks. Yet, because existing methods compare the extrinsic geometry of representations in state space, rather than their intrinsic geometry, they may fail to capture subtle yet crucial distinctions between fundamentally different neural network solutions. Here, we introduce metric similarity analysis (MSA), a novel method which leverages tools from Riemannian geometry to compare the intrinsic geometry of neural representations under the manifold hypothesis. We show that MSA can be used to i) disentangle features of neural computations in deep networks with different learning regimes, ii) compare nonlinear dynamics, and iii) investigate diffusion models. Hence, we introduce a mathematically grounded and broadly applicable framework to understand the mechanisms behind neural computations by comparing their intrinsic geometries.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。