让神经元像生物一样自主通信,构建图结构网络提升模型性能。
Intelligent Neural Networks: From Layered Architectures to Graph-Organized Intelligence
- 神经元自带记忆和通信路由,以完整图结构代替传统层叠架构。
- 在Text8任务上达到1.705 BPC,优于Transformer的2.055 BPC。
- 图结构设计增强训练稳定性,适合追求可解释与可扩展架构的研究者。
生物神经元具备内态维持、选择性通信和自组织成复杂图结构的能力。本文提出智能神经网络(INN),将神经元作为具有内部记忆和学习通信模式的一等实体,以完全图而非顺序层组织。每个智能神经元结合选择性状态空间动态与基于注意力的路由机制,通过图结构交互实现涌现计算。在Text8字符建模基准上,INN达到1.705比特/字符(BPC),显著优于同规模Transformer(2.055 BPC),并接近高度优化的LSTM基线。关键的是,在相同训练协议下,参数匹配的堆叠Mamba块无法收敛(>3.4 BPC),证明图拓扑对训练稳定性至关重要。消融实验表明,移除神经元间通信会导致性能下降或不稳定,验证了学习路由的价值。该工作表明,以神经元为中心的图组织不仅是生物启发,更是有效计算范式,为模块化、可解释与可扩展架构开辟新方向。
原文摘要 · Abstract (English)
Biological neurons exhibit remarkable intelligence: they maintain internal states, communicate selectively with other neurons, and self-organize into complex graphs rather than rigid hierarchical layers. What if artificial intelligence could emerge from similarly intelligent computational units? We introduce Intelligent Neural Networks (INN), a paradigm shift where neurons are first-class entities with internal memory and learned communication patterns, organized in complete graphs rather than sequential layers. Each Intelligent Neuron combines selective state-space dynamics (knowing when to activate) with attention-based routing (knowing to whom to send signals), enabling emergent computation through graph-structured interactions. On the standard Text8 character modeling benchmark, INN achieves 1.705 Bit-Per-Character (BPC), significantly outperforming a comparable Transformer (2.055 BPC) and matching a highly optimized LSTM baseline. Crucially, a parameter-matched baseline of stacked Mamba blocks fails to converge (>3.4 BPC) under the same training protocol, demonstrating that INN's graph topology provides essential training stability. Ablation studies confirm this: removing inter-neuron communication degrades performance or leads to instability, proving the value of learned neural routing. This work demonstrates that neuron-centric design with graph organization is not merely bio-inspired -- it is computationally effective, opening new directions for modular, interpretable, and scalable neural architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。