提出自适应记忆结晶机制,让AI在持续学习中既学新知识又不丢旧记忆。
Adaptive Memory Crystallization for Autonomous AI Agent Learning in Dynamic Environments
- 借鉴突触标记理论,用三阶段记忆架构实现经验从易变到稳定渐进固化
- 在多个基准上实现43%正向迁移提升、80%遗忘率下降和62%内存减少
- 适合需要长期持续学习的智能体系统,如机器人控制与多任务强化学习
在动态环境中运行的自主AI智能体面临持续学习的核心挑战:获得新能力的同时不丢失已有知识。本文提出自适应记忆结晶(AMC)机制,用于持续强化学习中的渐进式经验固化。AMC受突触标记与捕获(STC)理论启发,将记忆视为一个连续结晶过程,经验依据多目标效用信号从易变状态迁移到稳定状态。该框架构建了液态-玻璃态-晶体态三级记忆层级,由伊藤随机微分方程(SDE)驱动,其群体行为由显式福克-普朗克方程描述,具有闭式贝塔分布的稳态解。我们证明了:(i) 结晶SDE的适定性及全局收敛至唯一贝塔稳态分布;(ii) 个体结晶状态指数收敛至固定点,且给出明确收敛速率与方差界;(iii) 端到端Q学习误差界与匹配的记忆容量下界,直接关联SDE参数与智能体性能。在Meta-World MT50、Atari 20游戏序列学习与MuJoCo持续运动任务上的实验表明,AMC consistently 在正向迁移上提升34–43%,灾难性遗忘降低67–80%,内存占用减少62%。
原文摘要 · Abstract (English)
Autonomous AI agents operating in dynamic environments face a persistent challenge: acquiring new capabilities without erasing prior knowledge. We present Adaptive Memory Crystallization (AMC), a memory architecture for progressive experience consolidation in continual reinforcement learning. AMC is conceptually inspired by the qualitative structure of synaptic tagging and capture (STC) theory, the idea that memories transition through discrete stability phases, but makes no claim to model the underlying molecular or synaptic mechanisms. AMC models memory as a continuous crystallization process in which experiences migrate from plastic to stable states according to a multi-objective utility signal. The framework introduces a three-phase memory hierarchy (Liquid--Glass--Crystal) governed by an Itô stochastic differential equation (SDE) whose population-level behavior is captured by an explicit Fokker--Planck equation admitting a closed-form Beta stationary distribution. We provide proofs of: (i) well-posedness and global convergence of the crystallization SDE to a unique Beta stationary distribution; (ii) exponential convergence of individual crystallization states to their fixed points, with explicit rates and variance bounds; and (iii) end-to-end Q-learning error bounds and matching memory-capacity lower bounds that link SDE parameters directly to agent performance. Empirical evaluation on Meta-World MT50, Atari 20-game sequential learning, and MuJoCo continual locomotion consistently shows improvements in forward transfer (+34--43\% over the strongest baseline), reductions in catastrophic forgetting (67--80\%), and a 62\% decrease in memory footprint.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。