用马尔可夫采样训练脉冲神经网络，实现高效动态控制。
Learning to Control Dynamical Agents via Spiking Neural Networks and Metropolis-Hastings Sampling
- 基于马尔可夫采样迭代更新参数，无需梯度传播。
- 在AcroBot和CartPole上超越DQL与现有SNN方法，奖励更高、资源更少。
- 适合部署于类脑硬件的实时控制任务，能效优势显著。
脉冲神经网络（SNN）为实时控制系统提供了类生物、低功耗的替代方案，但其训练面临挑战，尤其在强化学习任务中，因脉冲通信不可微而难以使用传统梯度方法。本文提出首个基于马尔可夫-哈斯汀采样（Metropolis-Hastings）的SNN训练框架，用于动态智能体控制，无需依赖梯度。该方法通过累积奖励信号迭代生成并概率性接受参数更新，在不使用反向传播的前提下，实现了对神经形态平台的直接优化。我们在两个标准控制基准测试——AcroBot与CartPole上进行了评估。结果表明，该方法在最大化累积奖励的同时，显著降低网络资源消耗与训练轮次，优于传统深度Q学习（DQL）基线及先前的SNN强化学习方法。
原文摘要 · Abstract (English)
Spiking Neural Networks (SNNs) offer biologically inspired, energy-efficient alternatives to traditional Deep Neural Networks (DNNs) for real-time control systems. However, their training presents several challenges, particularly for reinforcement learning (RL) tasks, due to the non-differentiable nature of spike-based communication. In this work, we introduce what is, to our knowledge, the first framework that employs Metropolis-Hastings (MH) sampling, a Bayesian inference technique, to train SNNs for dynamical agent control in RL environments without relying on gradient-based methods. Our approach iteratively proposes and probabilistically accepts network parameter updates based on accumulated reward signals, effectively circumventing the limitations of backpropagation while enabling direct optimization on neuromorphic platforms. We evaluated this framework on two standard control benchmarks: AcroBot and CartPole. The results demonstrate that our MH-based approach outperforms conventional Deep Q-Learning (DQL) baselines and prior SNN-based RL approaches in terms of maximizing the accumulated reward while minimizing network resources and training episodes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。