用不对称兴奋抑制网络实现更稳定的神经动态，提升模型性能与训练效率。
TIDE: Asymmetric Neural Circuits for Stabilized Temporal Inhibitory-Excitatory Dynamics
- 基于威尔逊-科万动力学与侧向抑制构建稳定神经动态
- 训练时间减少50%以上，ImageNet准确率平均提升1.65%
- 符合生物真实，支持端到端训练，适合追求高效可靠的神经架构研究
近期的连续思维机器架构通过神经动力学将内部计算与外部输入解耦，但依赖多层感知机且缺乏稳定性保证。本文提出使用不对称兴奋-抑制（E-I）网络建模神经动力学，可通过网络理论原理实现稳定，并以基于博弈论的损失函数优化能量系统。在此基础上，我们提出时序抑制-兴奋动态引擎（TIDE），一种受神经科学启发的架构，通过引入威尔逊-科万动力学和侧向抑制，稳定地计算内部表示。TIDE在生物合理性上有所体现，例如采用分层感受野并强制遵循戴尔定律，实现80:20的兴奋-抑制比例，且整个架构可端到端训练。本文给出了收敛性、稳定性及复杂度的证明，并进行了实证消融分析。总体而言，TIDE在训练时间低于50%的情况下超越连续思维机器，且在多种扰动下平均提升ImageNet top-1准确率1.65%。
原文摘要 · Abstract (English)
Recent Continuous Thought Machine architecture decouples internal computation from external inputs via neural dynamics, but relies on multi-layer perceptrons without stability guarantees. We propose to model neural dynamics using asymmetric Excitatory-Inhibitory (E-I) networks, which can be stabilized via principles from network theory and can be expressed as energy-based systems optimized through a game-theoretic loss. Building on this perspective, we introduce Temporal Inhibitory-Excitatory Dynamic Engine (TIDE), a neuro-inspired architecture that computes internal representations through neural dynamics stabilized by incorporating the Wilson-Cowan dynamics and lateral inhibition. TIDE balances biological realism by, for instance, using Hierarchical Receptive Fields and enforcing Dale's principle to ensure a realistic $80:20$ E-I balance ratio with an end-to-end trainable architecture. The aim of this paper is to introduce a new architecture that brings neuro-inspired learning to the forefront. We present proofs of convergence, stability, and complexity bounds, along with empirical ablation studies. Overall, TIDE surpasses CTM with under $50\%$ of the training time and improves $\texttt{top-1}$ accuracy by an average of $+1.65\%$ on ImageNet under various perturbations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。