通过精细稀疏分配与休眠神经元激活，提升强化学习持续训练的适应性与稳定性。
Mastering Continual Reinforcement Learning through Fine-Grained Sparse Network Allocation and Dormant Neuron Exploration
- 将参数分为可冻结与可训练部分，实现任务间高效知识迁移。
- 在CW10-v1基准上达到95%成功率，显著优于已有方法。
- 适合关注长期学习与灾难性遗忘问题的研究者。
持续强化学习（CRL）对构建能随时间学习、适应并积累知识的智能体至关重要。然而，智能体必须在可塑性（快速习得新技能）与稳定性（长期保留知识，避免灾难性遗忘）之间取得平衡。本文提出SSDE，一种基于结构的方法，通过细粒度稀疏分配与休眠引导探索增强可塑性。SSDE将参数空间分解为前向迁移（冻结）参数与任务特定（可训练）参数，采用高效的共分配机制，在稀疏编码下确保新任务有足够的可训练容量，同时通过冻结参数促进高效前向迁移。然而，基于结构的方法常因非可训练参数累积而产生僵化，限制探索与适应能力。为此，本文进一步提出敏感性引导的神经元重激活机制，系统识别并重置推理中影响微弱的休眠神经元，有效增强探索能力，同时保持结构效率。在CW10-v1持续世界基准上的大量实验表明，SSDE达到95%的成功率，显著优于先前方法，在可塑性与稳定性权衡上表现更优（代码见：https://github.com/chengqiArchy/SSDE）。
原文摘要 · Abstract (English)
Continual Reinforcement Learning (CRL) is essential for developing agents that can learn, adapt, and accumulate knowledge over time. However, a fundamental challenge persists as agents must strike a delicate balance between plasticity, which enables rapid skill acquisition, and stability, which ensures long-term knowledge retention while preventing catastrophic forgetting. In this paper, we introduce SSDE, a novel structure-based approach that enhances plasticity through a fine-grained allocation strategy with Structured Sparsity and Dormant-guided Exploration. SSDE decomposes the parameter space into forward-transfer (frozen) parameters and task-specific (trainable) parameters. Crucially, these parameters are allocated by an efficient co-allocation scheme under sparse coding, ensuring sufficient trainable capacity for new tasks while promoting efficient forward transfer through frozen parameters. However, structure-based methods often suffer from rigidity due to the accumulation of non-trainable parameters, limiting exploration and adaptability. To address this, we further introduce a sensitivity-guided neuron reactivation mechanism that systematically identifies and resets dormant neurons, which exhibit minimal influence in the sparse policy network during inference. This approach effectively enhance exploration while preserving structural efficiency. Extensive experiments on the CW10-v1 Continual World benchmark demonstrate that SSDE achieves state-of-the-art performance, reaching a success rate of 95%, surpassing prior methods significantly in both plasticity and stability trade-offs (code is available at: https://github.com/chengqiArchy/SSDE).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。