将生物学习机制融入Mamba模型，实现高效长程依赖建模。
Bio-Inspired Mamba: Temporal Locality and Bioplausible Learning in Selective State Space Models
- 结合实时循环学习与类脉冲时序可塑性规则，实现在线学习。
- 在语言、语音和生物信号任务中性能媲美传统方法。
- 更节能，适合神经形态硬件，为脑科学提供新视角。
本文提出生物启发的Mamba（BIM），一种针对选择性状态空间模型的新型在线学习框架，融合了生物学习原理与Mamba架构。BIM将实时循环学习（RTRL）与类脉冲时序可塑性（STDP）的局部学习规则结合，解决了训练脉冲神经网络中时间局部性和生物合理性的问题。该方法利用时间反向传播与STDP之间的内在联系，提供了一种计算高效的替代方案，同时保持捕捉长程依赖的能力。我们在语言建模、语音识别和生物医学信号分析任务上评估BIM，结果表明其性能可与传统方法竞争，且符合生物学习原则。实验显示其具有更高的能效，具备神经形态硬件部署潜力。BIM不仅推动了生物合理机器学习的发展，还为生物神经网络中的时间信息处理机制提供了新见解。
原文摘要 · Abstract (English)
This paper introduces Bio-Inspired Mamba (BIM), a novel online learning framework for selective state space models that integrates biological learning principles with the Mamba architecture. BIM combines Real-Time Recurrent Learning (RTRL) with Spike-Timing-Dependent Plasticity (STDP)-like local learning rules, addressing the challenges of temporal locality and biological plausibility in training spiking neural networks. Our approach leverages the inherent connection between backpropagation through time and STDP, offering a computationally efficient alternative that maintains the ability to capture long-range dependencies. We evaluate BIM on language modeling, speech recognition, and biomedical signal analysis tasks, demonstrating competitive performance against traditional methods while adhering to biological learning principles. Results show improved energy efficiency and potential for neuromorphic hardware implementation. BIM not only advances the field of biologically plausible machine learning but also provides insights into the mechanisms of temporal information processing in biological neural networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。