arXiv:2603.15725cs.MAcs.ET2026-03

用脉冲神经网络实现轻量级机器人强化学习,部署高效且稳定。

S2Act: Simple Spiking Actor

  • 构建基于脉冲神经元的演员-评论家模型,通过近似率编码实现训练
  • 在多智能体随机环境中任务表现优于基线,推理延迟低
  • 适合边缘计算与真实机器人快速部署,减少超参数调优

脉冲神经网络(SNN)和生物启发式学习机制在移动机器人中具有吸引力,因其对机载神经网络策略的尺寸与性能受限于功耗和计算预算。现有方法如群体编码、奖励调制及混合人工神经网络(ANN)-SNN架构虽有成效,但在复杂高随机性环境中仍因SNN对超参数敏感和梯度信号不一致而受限。为此,我们提出简单脉冲演员(S2Act),一个计算轻量的框架,分三步部署基于SNN的强化学习策略:(1) 基于近似速率编码脉冲神经元构建演员-评论家模型;(2) 使用兼容激活函数进行梯度训练;(3) 将训练权重转换为速率编码漏电积分发放(LIF)神经元的物理参数以用于推理与部署。通过全局调整LIF神经元参数使其速率响应逼近ReLU激活,有效缓解梯度消失问题,同时预约束响应曲线降低对复杂SNN超参数调优的依赖。我们在两个多智能体随机环境(夺旗与停车)中验证该方法,模拟了多机器人交互的复杂性,并在真实TurtleBot平台上使用英特尔Loihi神经形态硬件部署训练策略。实验结果表明,S2Act在几乎所有场景下均优于相关基线,在任务表现和实时推理方面均有提升,展现出其在SNN强化学习策略快速原型设计与高效真实部署中的潜力。

原文摘要 · Abstract (English)

Spiking neural networks (SNNs) and biologically-inspired learning mechanisms are attractive in mobile robotics, where the size and performance of onboard neural network policies are constrained by power and computational budgets. Existing SNN approaches, such as population coding, reward modulation, and hybrid artificial neural network (ANN)-SNN architectures, have shown promising results; however, they face challenges in complex, highly stochastic environments due to SNN sensitivity to hyperparameters and inconsistent gradient signals. To address these challenges, we propose simple spiking actor (S2Act), a computationally lightweight framework that deploys an RL policy using an SNN in three steps: (1) architect an actor-critic model based on an approximated network of rate-based spiking neurons, (2) train the network with gradients using compatible activation functions, and (3) transfer the trained weights into physical parameters of rate-based leaky integrate-and-fire (LIF) neurons for inference and deployment. By globally shaping LIF neuron parameters such that their rate-based responses approximate ReLU activations, S2Act effectively mitigates the vanishing gradient problem, while pre-constraining LIF response curves reduces reliance on complex SNN-specific hyperparameter tuning. We demonstrate our method in two multi-agent stochastic environments (capture-the-flag and parking) that capture the complexity of multi-robot interactions, and deploy our trained policies on physical TurtleBot platforms using Intel's Loihi neuromorphic hardware. Our experimental results show that S2Act outperforms relevant baselines in task performance and real-time inference in nearly all considered scenarios, highlighting its potential for rapid prototyping and efficient real-world deployment of SNN-based RL policies.

脉冲神经网络强化学习机器人部署神经形态计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。