受小脑结构启发，提出高效鲁棒的强化学习新架构
CDRL: A Reinforcement Learning Framework Inspired by Cerebellar Circuits and Dendritic Computational Strategies
- 借鉴小脑结构设计大扩张稀疏连接网络
- 在噪声高维任务中样本效率提升显著
- 适合资源受限下的强化学习应用
强化学习在高维序列决策任务中表现优异，但面临样本效率低、对噪声敏感及部分可观测下泛化能力弱的问题。现有方法多依赖优化策略，而架构先验对表征学习与决策动态的影响尚未充分探索。受小脑结构原理启发，我们提出一种生物驱动的强化学习架构，包含大扩张、稀疏连接、稀疏激活和树突级调制。在噪声高维基准测试中，该架构与树突调制均显著提升样本效率、鲁棒性与泛化性能。参数敏感性分析表明，小脑式结构可在模型参数受限时实现最优表现。本工作强调了小脑结构先验作为强化学习有效归纳偏置的价值。
原文摘要 · Abstract (English)
Reinforcement learning (RL) has achieved notable performance in high-dimensional sequential decision-making tasks, yet remains limited by low sample efficiency, sensitivity to noise, and weak generalization under partial observability. Most existing approaches address these issues primarily through optimization strategies, while the role of architectural priors in shaping representation learning and decision dynamics is less explored. Inspired by structural principles of the cerebellum, we propose a biologically grounded RL architecture that incorporate large expansion, sparse connectivity, sparse activation, and dendritic-level modulation. Experiments on noisy, high-dimensional RL benchmarks show that both the cerebellar architecture and dendritic modulation consistently improve sample efficiency, robustness, and generalization compared to conventional designs. Sensitivity analysis of architectural parameters suggests that cerebellum-inspired structures can offer optimized performance for RL with constrained model parameters. Overall, our work underscores the value of cerebellar structural priors as effective inductive biases for RL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。