arXiv:2507.06326cs.LGcs.AI2025-07中稿 · IEEE IMC 2025被引 2

提出高效强化学习控制器,实现帕金森病深部脑刺激的自适应调控。

Sample-Efficient Reinforcement Learning Controller for Deep Brain Stimulation in Parkinson's Disease

  • 用预测奖励模型和软性探索机制提升样本效率与稳定性。
  • 在模拟中实现更快收敛,显著抑制病理性β波活动。
  • 适合资源受限的植入式神经调控设备部署使用。

深部脑刺激(DBS)是治疗帕金森病的有效手段,但传统开环系统缺乏自适应性,持续刺激导致能耗高,个性化不足。自适应DBS(aDBS)通过β频段振荡等生物标志物实现闭环调控。尽管强化学习(RL)有望实现个性化控制,但现有方法存在样本复杂度高、二元动作空间探索不稳、难以在资源受限硬件上部署等问题。本文提出SEA-DBS,一种高效的演员-评论家框架,集成预测奖励模型以减少对实时反馈的依赖,并采用基于Gumbel Softmax的探索策略,在二元动作空间中实现稳定可微的策略更新。该方法提升了样本效率、探索鲁棒性及对资源受限神经调控硬件的兼容性。我们在生物合理化的帕金森基底节活动仿真中评估了SEA-DBS,结果表明其收敛更快,病理β波功率抑制更强,且对训练后FP16量化具有鲁棒性。实验验证了SEA-DBS是一种实用高效的RL驱动aDBS框架,适用于实时、资源受限的神经调控场景。

原文摘要 · Abstract (English)

Deep brain stimulation (DBS) is an established intervention for Parkinson's disease (PD), but conventional open-loop systems lack adaptability, are energy-inefficient due to continuous stimulation, and provide limited personalization to individual neural dynamics. Adaptive DBS (aDBS) offers a closed-loop alternative, using biomarkers such as beta-band oscillations to dynamically modulate stimulation. While reinforcement learning (RL) holds promise for personalized aDBS control, existing methods suffer from high sample complexity, unstable exploration in binary action spaces, and limited deployability on resource-constrained hardware. We propose SEA-DBS, a sample-efficient actor-critic framework that addresses the core challenges of RL-based adaptive neurostimulation. SEA-DBS integrates a predictive reward model to reduce reliance on real-time feedback and employs Gumbel Softmax-based exploration for stable, differentiable policy updates in binary action spaces. Together, these components improve sample efficiency, exploration robustness, and compatibility with resource-constrained neuromodulatory hardware. We evaluate SEA-DBS on a biologically realistic simulation of Parkinsonian basal ganglia activity, demonstrating faster convergence, stronger suppression of pathological beta-band power, and resilience to post-training FP16 quantization. Our results show that SEA-DBS offers a practical and effective RL-based aDBS framework for real-time, resource-constrained neuromodulation.

强化学习深部脑刺激自适应调控神经工程

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。