让机器人持续学习,不遗忘旧任务且参数极少。
Dynamic Mixture of Progressive Parameter-Efficient Expert Library for Lifelong Robot Learning
- 构建可动态组合的低秩专家库,用轻量路由选择最佳策略。
- 在LIBERO数据集上连续学习成功率达90%以上,参数仅占0.1%。
- 适合需要长期适应新任务的机器人系统,存储效率极高。
通用智能体需在其生命周期内持续学习并适应,实现高效前向迁移的同时最小化灾难性遗忘。以往基于预训练-微调范式的参数高效微调方法虽能实现单任务适配,但依赖测试时任务标识,且各适配器间无法共享知识。为此,我们提出动态渐进式参数高效专家库(DMPEL),通过逐步构建低秩专家库,并使用轻量级路由器动态组合专家形成端到端策略,支持灵活高效的持续前向迁移。同时,利用微调参数的模块化结构,引入专家系数回放机制,引导路由器准确召回已学任务的冻结专家,有效缓解遗忘问题,且相比经验回放显著降低存储与计算开销。在终身机器人学习基准LIBERO上的大量实验表明,本框架在持续适应过程中成功率优于现有先进方法,同时仅使用极少量可训练参数和存储空间。
原文摘要 · Abstract (English)
A generalist agent must continuously learn and adapt throughout its lifetime, achieving efficient forward transfer while minimizing catastrophic forgetting. Previous work within the dominant pretrain-then-finetune paradigm has explored parameter-efficient fine-tuning for single-task adaptation, effectively steering a frozen pretrained model with a small number of parameters. However, in the context of lifelong learning, these methods rely on the impractical assumption of a test-time task identifier and restrict knowledge sharing among isolated adapters. To address these limitations, we propose Dynamic Mixture of Progressive Parameter-Efficient Expert Library (DMPEL) for lifelong robot learning. DMPEL progressively builds a low-rank expert library and employs a lightweight router to dynamically combine experts into an end-to-end policy, enabling flexible and efficient lifelong forward transfer. Furthermore, by leveraging the modular structure of the fine-tuned parameters, we introduce expert coefficient replay, which guides the router to accurately retrieve frozen experts for previously encountered tasks. This technique mitigates forgetting while being significantly more storage- and computation-efficient than experience replay over the entire policy. Extensive experiments on the lifelong robot learning benchmark LIBERO demonstrate that our framework outperforms state-of-the-art lifelong learning methods in success rates during continual adaptation, while utilizing minimal trainable parameters and storage.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。