受大脑记忆机制启发，新模型有效缓解强化学习中的知识遗忘问题。
NeuroSynth: A Biologically Inspired Continual Reinforcement Learning Architecture for Mitigating Catastrophic Forgetting
- 采用海马体-皮层双路径机制分离快速学习与长期记忆。
- 在三阶段导航任务中，保留早期任务成功率达18%以上。
- 适合需要持续学习的智能体系统设计，如机器人控制。
人工智能系统在单一任务上表现良好，但在持续学习中易因新任务覆盖旧知识而出现灾难性遗忘。本文提出神经合成（NeuroSynth），一种受生物记忆机制启发的持续强化学习架构，通过双路径整合机制实现快速学习与长期巩固。该方法区分‘计划’与‘习惯’两条路径，并结合重放与知识蒸馏。在三个顺序导航任务（目标位置变化）且不重复访问的持续学习设置下，与PPO和EWC对比评估。六次独立随机种子实验显示，训练后NeuroSynth对任务A的保留成功率为18.00%，远超PPO的0.33%（p=0.014929, Cohen's d=1.49）；任务B成功率为35.33%，而PPO为0.00%（p=0.002376, Cohen's d=2.31）。最终任务C性能优于EWC（9.00% vs 2.00%，p=0.226643, Cohen's d=0.56），优势虽未达统计显著但存在。结果表明，生物启发的巩固机制有助于提升持续强化学习系统的稳定性与可塑性平衡。
原文摘要 · Abstract (English)
Artificial Intelligence (AI) systems often perform well on isolated tasks but struggle under continual learning conditions, where training on new tasks can overwrite previously acquired knowledge, a failure mode known as catastrophic forgetting. Biological learning systems reduce this interference through complementary memory processes involving rapid hippocampal encoding and slower cortical consolidation. This study introduces NeuroSynth, a brain-inspired continual reinforcement learning architecture designed to mitigate catastrophic forgetting through a dual-pathway consolidation mechanism. NeuroSynth separates rapid task acquisition from long-term retention using distinct "plan" and "habit" pathways combined with replay and knowledge distillation. NeuroSynth was evaluated against Proximal Policy Optimization (PPO) and Elastic Weight Consolidation (EWC) across three sequential navigation tasks with changing goal locations in a non-revisitation continual learning setting. Across six independent seeds, NeuroSynth preserved substantially more early-task knowledge than PPO after sequential training, achieving 18.00% Task A success rate compared to 0.33% for PPO (p = 0.014929, Cohen's d = 1.49) and 35.33% Task B success rate compared to 0.00% for PPO (p = 0.002376, Cohen's d = 2.31). NeuroSynth also demonstrated higher final Task C performance than EWC, achieving 9.00% compared to 2.00% (p = 0.226643, Cohen's d = 0.56), indicating a moderate but not statistically significant advantage. These findings suggest that biologically inspired consolidation mechanisms may improve the stability-plasticity balance in continual reinforcement learning systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。