让机器人通过经验自动优化技能库，持续提升任务完成能力。
PRACTICE: From Experience to Expertise in Self-Evolving Embodied Agents
- 用自学习机制从过往经验中发现并维护技能库，无需人工设计提示流程。
- 在EB-ALFRED和EB-Habitat上表现优于现有最先进方法，性能随更新轮次持续提升。
- 适合需要长期适应环境的智能体系统，尤其适用于固定执行器场景。
近期研究显示，多模态大语言模型（MLLMs）可作为具身智能体，将语言指令与视觉观察转化为可执行计划。然而，构建能通过交互持续改进并快速适应环境的智能体仍具挑战。总结过往交互轨迹的经验提供了一条可行路径，但现有基于经验的方法常依赖人工设计的提示流程来提取和更新技能，此类固定流程难以从新而多样的经验中学习新技能。本文提出PRACTICE，训练一个技能学习器从历史交互轨迹中发现并维护持久的技能库，同时保持任务执行器不变。该学习器根据累积技能与新轨迹生成结构化批量编辑（增删、合并、修正），并层级整合所有编辑形成一致更新后的技能库。训练采用两阶段课程：首先在理想轨迹上学习基础技能生成与库维护；其次通过对比同一任务下不同执行器的成功与失败轨迹，学会识别无效动作模式与恢复策略。最后，采用在线技能编辑蒸馏，使学习器与更强教师对齐当前编辑分布，进一步优化策略。实验表明，紧凑的技能学习器在多轮库更新中为多个固定执行器带来持续性能提升。在EB-ALFRED与EB-Habitat上，PRACTICE超越最强基线方法。项目资源已公开：https://baai-agents.github.io/PRACTICE
原文摘要 · Abstract (English)
Recent studies have shown that multimodal large language models (MLLMs) can serve as embodied agents, translating language instructions and visual observations into executable plans. However, building agents that can continually improve through interaction and rapidly adapt to their environments remains challenging. Summing up experience from past interaction trajectories provides a promising solution, but existing experience-based methods often rely on manually designed prompting workflows to extract and update skills. Such fixed procedures may struggle to learn updated skills from new and diverse experiences. We introduce PRACTICE, which trains a skill learner to discover and maintain a persistent skill library from past interaction trajectories while keeping the task executor frozen. Given the historical accumulated skills and incoming trajectories, the skill learner produces structured batch-edits that add, refine, merge, or remove skills, and then hierarchical consolidate all collected edits into a consistent updated skill library. We train the learner with a two-stage curriculum. First, it learns basic skill generation and library maintenance from oracle trajectories. Then, by contrasting successful and failed trajectories from heterogeneous executors on the same tasks, it learn to identify invalid action patterns and recovery strategies. Finally, we apply online skill-edit distillation to align the skill learner with a stronger teacher on its current edit distribution to further improves the policy. Experiments demonstrate that a compact skill learner delivers consistent performance improvements across successive library-update rounds for multiple frozen executors. On EB-ALFRED and EB-Habitat, PRACTICE further outperforms the strongest experience-based baselines. Project resources are publicly available at: https://baai-agents.github.io/PRACTICE
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。