模型的局部内在维度越低，越能更好泛化并匹配人脑活动。
Local Intrinsic Dimension of Representations Predicts Alignment and Generalization in AI Models and Human Brain
- 用局部内在维度衡量表示的几何特性
- 维度越低，模型间与模型-脑对齐越强，泛化越好
- 适合研究模型泛化与脑认知的交叉领域
近期研究表明，泛化能力更强的神经网络在不同架构和训练方式下表现出更高的表示对齐性。本文发现，泛化能力强的模型也与人类神经活动具有更强的对齐性。此外，模型泛化性能、模型间对齐性以及模型与大脑对齐性三者显著相关。我们进一步证明，这些关系可由学习表示的一个单一几何属性——嵌入的局部内在维度解释：局部维度越低，模型间对齐、模型与大脑对齐及泛化性能均越强；而全局维度无法捕捉此类效应。最后，我们发现增大模型容量和训练数据规模会系统性降低局部内在维度，为扩展带来的优势提供了几何解释。结果表明，局部内在维度是人工与生物系统中表示收敛性的统一描述符。
原文摘要 · Abstract (English)
Recent work has found that neural networks with stronger generalization tend to exhibit higher representational alignment with one another across architectures and training paradigms. In this work, we show that models with stronger generalization also align more strongly with human neural activity. Moreover, generalization performance, model--model alignment, and model--brain alignment are all significantly correlated with each other. We further show that these relationships can be explained by a single geometric property of learned representations: the local intrinsic dimension of embeddings. Lower local dimension is consistently associated with stronger model--model alignment, stronger model--brain alignment, and better generalization, whereas global dimension measures fail to capture these effects. Finally, we find that increasing model capacity and training data scale systematically reduces local intrinsic dimension, providing a geometric account of the benefits of scaling. Together, our results identify local intrinsic dimension as a unifying descriptor of representational convergence in artificial and biological systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。