固定结构的神经网络可通过输入动态实现不同计算模式,提升能效与性能。
Dynamical Alignment: A Principle for Adaptive Neural Computation
- 通过输入动态调控网络计算模式,突破静态结构限制。
- 收缩动力学模式实现稀疏编码,能效更高;扩张模式提升表征能力。
- 适用于脑启发脉冲网络优化,对AI架构设计有范式启示。
神经网络的计算能力通常被认为由其静态结构决定。本文挑战这一观点,提出固定结构的网络可因输入信号的时间动态而进入根本不同的计算模式,这一机制称为「动态对齐」。该原理为长期存在的脉冲神经网络(SNNs)性能不足问题提供了新解:将静态输入编码为可控的动力学轨迹,揭示了一个双模优化景观,其关键相变由相空间体积动态决定。在由收缩动力学驱动的『耗散』模式中,网络通过稀疏时间编码实现更优能效;而在由扩展动力学驱动的『扩张』模式中,其表征能力足以使SNN在分类、强化学习及认知整合等任务上达到甚至超越人工神经网络水平。这种计算优势源于输入动态与神经元积分时间尺度的对齐。该原理统一解释了神经科学中长期观察到的二元性,如稳定性-可塑性矛盾与分离-整合动态。它表明生物与人工系统中的计算可通过‘软件’在固定‘硬件’上动态塑造,暗示人工智能研究可能从复杂静态架构转向掌握自适应动态计算原则。
原文摘要 · Abstract (English)
The computational capabilities of a neural network are widely assumed to be determined by its static architecture. Here we challenge this view by establishing that a fixed neural structure can operate in fundamentally different computational modes, driven not by its structure but by the temporal dynamics of its input signals. We term this principle 'Dynamical Alignment'. Applying this principle offers a novel resolution to the long-standing paradox of why brain-inspired spiking neural networks (SNNs) underperform. By encoding static input into controllable dynamical trajectories, we uncover a bimodal optimization landscape with a critical phase transition governed by phase space volume dynamics. A 'dissipative' mode, driven by contracting dynamics, achieves superior energy efficiency through sparse temporal codes. In contrast, an 'expansive' mode, driven by expanding dynamics, unlocks the representational power required for SNNs to match or even exceed their artificial neural network counterparts on diverse tasks, including classification, reinforcement learning, and cognitive integration. We find this computational advantage emerges from a timescale alignment between input dynamics and neuronal integration. This principle, in turn, offers a unified, computable perspective on long-observed dualities in neuroscience, from stability-plasticity dilemma to segregation-integration dynamic. It demonstrates that computation in both biological and artificial systems can be dynamically sculpted by 'software' on fixed 'hardware', pointing toward a potential paradigm shift for AI research: away from designing complex static architectures and toward mastering adaptive, dynamic computation principles.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。