arXiv:2605.14211cs.AIcs.LG2026-05

AI agent通过自学习从网络视频中提升长时任务能力,无需人工标注。

ASH: Agents that Self-Hone via Embodied Learning

论文配图:ASH: Agents that Self-Hone via Embodied Learning
图 1 · 摘自论文原文
  • 自建改进循环:用自身轨迹学逆动力模型,从网络视频提取训练信号
  • 在两款游戏中分别达成11.2/12和9.9/12里程碑,远超基线的6.5/12和6.0/12
  • 适合研究长时序决策与无监督强化学习的学者

长时程具身任务仍是人工智能的核心挑战,现有方法依赖人工设计奖励或动作标注示范,均难扩展。本文提出ASH,一种可从无标签、含噪互联网视频中自主学习具身策略的智能体系统,无需奖励塑形或专家标注。ASH采用自我改进循环:当陷入困境时,它基于自身轨迹学习逆动力模型(IDM),并利用该模型从相关互联网视频中提取监督信号。通过无监督学习识别大规模视频中的关键时刻,并将其作为长期记忆保留,使系统能应对长时程任务。我们在两个互补环境中评估:需要数小时规划的《宝可梦:绿宝石》(回合制角色扮演)与《塞尔达传说:缩小帽》(实时动作冒险)。在8小时评估中,行为克隆、检索增强及零样本基础模型基线均停滞不前,而ASH持续推进。在《宝可梦:绿宝石》中平均达成11.2/12个里程碑,在《塞尔达传说》中为9.9/12;最强基线分别仅达6.5/12和6.0/12。结果表明,自改进智能体是长时程具身学习的可扩展路径。

原文摘要 · Abstract (English)

Long-horizon embodied tasks remain a fundamental challenge in AI, as current methods rely on hand-engineered rewards or action-labeled demonstrations, neither of which scales. We introduce ASH, an agentic system that learns an embodied policy from unlabeled, noisy internet video, without reward shaping or expert annotation. ASH follows a self-improvement loop; when it gets stuck, ASH learns an Inverse Dynamics Model (IDM) from its own trajectories, and uses its IDM to extract supervision from relevant internet video. ASH uses unsupervised learning to identify key moments from large-scale internet video and retains them as long-term memory -- allowing it to tackle long-horizon problems. We evaluate ASH on two complementary environments demanding multi-hour planning: Pokemon Emerald, a turn-based RPG, and The Legend of Zelda: The Minish Cap, a real-time action-adventure game. In both games, behavioral cloning, retrieval-augmented and zero-shot foundation-model baselines plateau, while ASH sustains progression across our 8-hour evaluation. ASH reaches an average of $11.2/12$ milestones in Pokemon Emerald and $9.9/12$ in Legend of Zelda, while the strongest baseline gets stuck in both environments at an average of $6.5/12$ and $6.0/12$ milestones, respectively. We demonstrate that self-improving agents are a scalable recipe for long-horizon embodied learning.

具身智能自学习长时序任务无监督学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。