RNN通过动态扭曲表示来完成任务，揭示其计算本质。
RNNs perform task computations by dynamically warping neural representations
- 用黎曼几何框架分析RNN动态表示变化
- 发现RNN计算本质是动态扭曲神经表示
- 适合研究模型可解释性与动态系统者阅读
分析神经网络激活中数据特征的表示有助于理解其任务执行机制。长期以来，研究聚焦于数学刻画这些“神经表示”的几何结构。与此同时，机器学习领域对动态系统如何处理时变输入数据的兴趣激增。然而，计算-动力学与表示几何之间的关联仍不清晰。本文提出假设：循环神经网络（RNN）通过动态扭曲任务变量的表示来完成计算。为此，我们构建了一个黎曼几何框架，能够从输入流形推导出动态系统的流形拓扑与几何结构。通过对RNN时间演化几何的刻画，我们证明动态扭曲是其计算的核心特征。
原文摘要 · Abstract (English)
Analysing how neural networks represent data features in their activations can help interpret how they perform tasks. Hence, a long line of work has focused on mathematically characterising the geometry of such "neural representations." In parallel, machine learning has seen a surge of interest in understanding how dynamical systems perform computations on time-varying input data. Yet, the link between computation-through-dynamics and representational geometry remains poorly understood. Here, we hypothesise that recurrent neural networks (RNNs) perform computations by dynamically warping their representations of task variables. To test this hypothesis, we develop a Riemannian geometric framework that enables the derivation of the manifold topology and geometry of a dynamical system from the manifold of its inputs. By characterising the time-varying geometry of RNNs, we show that dynamic warping is a fundamental feature of their computations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。