强任务下神经网络自发出现相似关键特征轴。
Contravariance Theory: Strong Alignment for Minimal Solutions to Hard Tasks
- 用仿射映射证明最小解间表示对齐可导出特权轴强对齐。
- 任务越难，网络层级越自发形成共享的特权特征方向。
- 适合研究脑与模型类比、神经网络共演化机制的学者。
过去十五年神经人工智能的研究揭示了两大核心问题：如何比较深度神经网络与大脑之间的关系，以及人工网络与真实脑网络之间是否会出现趋同演化。本文表明，在足够困难的任务下，任意两个最小化深度神经网络解具有如下特性：(i) 基于仿射映射的“弱”表示对齐可保证“强”特权轴对齐；(ii) 对齐效应沿网络层次“拉链式”推进，由端到端任务优化催生出特权轴。这些结果形式化了Cao与Yamins [2024] 提出的反变性（contravariance）概念，并揭示了神经人工智能理论的重要启示：在足够强的任务下，网络间比较所依赖的度量选择不敏感，且趋同演化几乎是不可避免的。
原文摘要 · Abstract (English)
A series of results from the NeuroAI over the past fifteen years have raised core questions both about how to compare Deep Neural Network (DNN) models to the brain, and about how much convergent evolution to expect between artificial networks and real brain networks. Here, we show that for any two minimal DNN solutions to a sufficiently hard task: (i) "weak" alignment of network representations based on affine mappings guarantees "strong" alignment of privileged axes, and (ii) alignment "zippers" up the network hierarchy, causing the emergence of privileged axes from end-to-end task optimization. These results formalize the notion of contravariance from Cao and Yamins [2024], and illustrate important consequences for the theory of NeuroAI: with sufficiently strong tasks, choice of metric for inter-network comparison is not all that sensitive, and that convergent evolution is probably inevitable.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。