用视觉世界模型生成分阶段动作计划，让机器人更智能地完成复杂操作。
Act2Goal: From World Model To General Goal-conditioned Policy
- 通过视觉世界模型生成中间状态序列，规划长程操作路径。
- 多尺度时间哈希将计划分解为精细与宏观控制帧，提升执行稳定性。
- 无需奖励信号即可自主优化，实机任务成功率从30%提至90%。
指定机器人操作任务时，如何兼顾表达力与精确性仍是核心挑战。虽然视觉目标能提供简洁明确的任务描述，但现有目标条件策略常因仅依赖单步动作预测，缺乏对任务进展的显式建模，导致长程操作表现不佳。本文提出Act2Goal，一种通用的目标条件操作策略，结合目标条件视觉世界模型与多尺度时间控制机制。给定当前观测和目标视觉状态，世界模型生成一系列合理的中间视觉状态，捕捉长程结构。为将该视觉计划转化为稳健执行，我们引入多尺度时间哈希（MSTH），将想象轨迹分解为密集近端帧以实现细粒度闭环控制，以及稀疏远端帧以维持全局任务一致性。策略通过端到端跨注意力机制将这些表示与运动控制耦合，实现连贯的长程行为，同时对局部扰动保持响应性。Act2Goal在新物体、空间布局及环境中表现出强零样本泛化能力。进一步通过基于LoRA的回溯目标重标注实现免奖励在线适应，支持无外部监督下的快速自主改进。真实机器人实验表明，仅数分钟自主交互后，其在困难分布外任务上的成功率从30%提升至90%，验证了目标条件世界模型结合多尺度时间控制可提供长程操作所需的结构化引导。
原文摘要 · Abstract (English)
Specifying robotic manipulation tasks in a manner that is both expressive and precise remains a central challenge. While visual goals provide a compact and unambiguous task specification, existing goal-conditioned policies often struggle with long-horizon manipulation due to their reliance on single-step action prediction without explicit modeling of task progress. We propose Act2Goal, a general goal-conditioned manipulation policy that integrates a goal-conditioned visual world model with multi-scale temporal control. Given a current observation and a target visual goal, the world model generates a plausible sequence of intermediate visual states that captures long-horizon structure. To translate this visual plan into robust execution, we introduce Multi-Scale Temporal Hashing (MSTH), which decomposes the imagined trajectory into dense proximal frames for fine-grained closed-loop control and sparse distal frames that anchor global task consistency. The policy couples these representations with motor control through end-to-end cross-attention, enabling coherent long-horizon behavior while remaining reactive to local disturbances. Act2Goal achieves strong zero-shot generalization to novel objects, spatial layouts, and environments. We further enable reward-free online adaptation through hindsight goal relabeling with LoRA-based finetuning, allowing rapid autonomous improvement without external supervision. Real-robot experiments demonstrate that Act2Goal improves success rates from 30% to 90% on challenging out-of-distribution tasks within minutes of autonomous interaction, validating that goal-conditioned world models with multi-scale temporal control provide structured guidance necessary for robust long-horizon manipulation. Project page: https://act2goal.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。