arXiv:2602.21198cs.LGcs.AI2026-02被引 9

让机器人像人一样从错误中学习,提升长期任务执行能力。

Learning from Trials and Errors: Reflective Test-Time Planning for Embodied LLMs

  • 引入测试时反思机制,分三步优化决策:即时反思、事后训练、回溯修正。
  • 在多个真实与仿真环境中实现显著性能提升,零样本泛化至真实机器人。
  • 适合研究具身智能、长程任务规划及人类反馈强化学习的开发者。

具身大模型赋予机器人高层任务推理能力,但缺乏对错误的反思能力,导致部署过程变成一系列独立试错,错误重复发生而非积累经验。受人类反思实践启发,我们提出反射式测试时规划,整合三种反思模式:行动中反思(test-time scaling)——在执行前通过内部反思生成并评估多个候选动作;行动后反思(test-time training)——基于外部反馈更新内部反思模型与动作策略;以及回溯性反思,使代理能回顾早期决策并结合后见之明进行模型更新,实现更优的长期信用分配。在自建的长周期家庭任务基准和MuJoCo柜子装配基准上的实验显示,该方法显著优于基线模型,且可在无训练条件下泛化至照片级真实环境(HM3D),并在Franka Panda机械臂上完成真实机器人实验。消融实验表明,三种反思机制相互依赖,回溯反思相比逐步外部反馈在更低计算开销下实现更优信用分配。定性分析进一步展示了反思带来的行为纠正效果。

原文摘要 · Abstract (English)

Embodied LLMs endow robots with high-level task reasoning, but they cannot reflect on what went wrong or why, turning deployment into a sequence of independent trials where mistakes repeat rather than accumulate into experience. Drawing upon human reflective practitioners, we introduce Reflective Test-Time Planning, which integrates two modes of reflection: \textit{reflection-in-action}, where the agent uses test-time scaling to generate and score multiple candidate actions using internal reflections before execution; and \textit{reflection-on-action}, which uses test-time training to update both its internal reflection model and its action policy based on external reflections after execution. We also include retrospective reflection, allowing the agent to re-evaluate earlier decisions and perform model updates with hindsight for proper long-horizon credit assignment. Experiments on our newly-designed Long-Horizon Household benchmark and MuJoCo Cupboard Fitting benchmark show significant gains over baseline models, with zero-shot generalization to photorealistic HM3D environments and real-robot experiments on a Franka Panda arm. Ablations confirm that reflection-in-action and reflection-on-action are mutually dependent, and that retrospective reflection achieves better credit assignment than step-wise external feedback at lower computational overhead. Qualitative analyses further highlight behavioral correction through reflection.

具身智能反思机制长程规划测试时学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。