Transformer模型在物理模拟中自发出现碰撞检测结构，其形成受损失曲面退化性调控。
Emergence of Computational Structure in a Neural Network Physics Simulator
- 通过注意力头学习粒子碰撞检测，实现可解释的计算结构
- 结构涌现与损失景观退化几何直接相关，动力学遵循幂律规律
- 为探测神经网络中的计算结构提供新方法，适合关注可解释性的研究者
神经网络常具备可识别的计算结构——执行特定可解释任务的组件，但这些结构如何产生及最佳检测方法尚不明确。本文研究了一个类似Transformer的模型在模拟粒子系统物理行为时计算结构的涌现过程，其中注意力机制用于粒子间信息传递。结果表明：(a) 变压器的注意力头会自发形成检测粒子碰撞的结构；(b) 这种结构的出现与损失景观中的退化几何密切相关；(c) 其涌现动态符合幂律规律。这暗示这些组件受退化的“有效势能”支配。研究结果对理解神经网络中计算结构的收敛时间有重要意义，并提出可通过分析网络组件动态来检测计算结构。
原文摘要 · Abstract (English)
Neural networks often have identifiable computational structures - components of the network which perform an interpretable algorithm or task - but the mechanisms by which these emerge and the best methods for detecting these structures are not well understood. In this paper we investigate the emergence of computational structure in a transformer-like model trained to simulate the physics of a particle system, where the transformer's attention mechanism is used to transfer information between particles. We show that (a) structures emerge in the attention heads of the transformer which learn to detect particle collisions, (b) the emergence of these structures is associated to degenerate geometry in the loss landscape, and (c) the dynamics of this emergence follows a power law. This suggests that these components are governed by a degenerate "effective potential". These results have implications for the convergence time of computational structure within neural networks and suggest that the emergence of computational structure can be detected by studying the dynamics of network components.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。