arXiv:2410.24070cs.LGcs.AI2024-10被引 3

提出动态相似性分析法,精准捕捉RNN中随学习演化的组合性动态特征。

Dynamical similarity analysis can identify compositional dynamics developing in RNNs

  • 构建基于吸引子和RNN的测试案例,验证度量方法对演化动态的识别能力。
  • DSA在噪声鲁棒性上优于Procrustes和CKA,更可靠地识别行为相关表征。
  • 适用于新架构如Mamba的分析,揭示其动态变化与传统RNN本质差异。

神经网络表示分析已成为神经科学与可解释性研究的重要工具。通过比较不同条件、架构及物种下神经元激活的相似性,可实现对信息转换过程的规模化理解。然而,现有部分度量指标易受虚假信号干扰,导致误导性结果。为筛选可靠度量并改进其性能,亟需设计具有明确基准意义的测试案例。本文提出利用循环神经网络(RNN)中的组合性学习现象作为测试基准,构建吸引子与RNN两类测试案例。实验表明,所提出的动态相似性分析(DSA)能有效识别学习过程中逐渐发展的表征,并更可靠地捕捉与网络计算相关的动态模式。相比经典方法Procrustes与CKA,DSA具备更强的抗噪能力。进一步将该方法应用于现代状态空间模型(Mamba),发现其因表达能力强,可能不随训练改变递归动态,与传统RNN形成对比。本工作通过构建可复现的测试案例,显著提升了对RNN中计算演化过程的理解。

原文摘要 · Abstract (English)

Methods for analyzing representations in neural systems have become a popular tool in both neuroscience and mechanistic interpretability. Having measures to compare how similar activations of neurons are across conditions, architectures, and species, gives us a scalable way of learning how information is transformed within different neural networks. In contrast to this trend, recent investigations have revealed how some metrics can respond to spurious signals and hence give misleading results. To identify the most reliable metric and understand how measures could be improved, it is going to be important to identify specific test cases which can serve as benchmarks. Here we propose that the phenomena of compositional learning in recurrent neural networks (RNNs) allows us to build a test case for dynamical representation alignment metrics. By implementing this case, we show it enables us to test whether metrics can identify representations which gradually develop throughout learning and probe whether representations identified by metrics are relevant to computations executed by networks. By building both an attractor- and RNN-based test case, we show that the new Dynamical Similarity Analysis (DSA) is more noise robust and identifies behaviorally relevant representations more reliably than prior metrics (Procrustes, CKA). We also show how test cases can be used beyond evaluating metrics to study new architectures. Specifically, results from applying DSA to modern (Mamba) state space models, suggest that, in contrast to RNNs, these models may not exhibit changes to their recurrent dynamics due to their expressiveness. Overall, by developing test cases, we show DSA's exceptional ability to detect compositional dynamical motifs, thereby enhancing our understanding of how computations unfold in RNNs.

动态表征RNN分析可解释性模型对比

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。