让机器人模型学会预测失败后果，提升动作生成可靠性。
FACT: Failure-Aware Causal Training for World-Action Models
- 基于动作条件的因果训练，让模型预测执行动作后的未来结果。
- 引入失败数据作为监督信号，使模型在坏动作下仍能准确预测任务进展。
- 适用于需要高可靠性的双臂操作场景，尤其适合真实世界部署。
近期的世界-动作模型（WAMs）表明，联合训练策略与未来预测可为动作生成提供物理先验。许多WAMs利用视频模型的未来预测能力，通过逆动力学模型恢复动作，或以预测视频作为动作生成的目标条件。然而，这些模型主要在成功示范数据上训练，难以预测糟糕动作的后果。本文提出FACT，一种因果型世界-动作模型，能够根据执行的动作预测未来视频和任务进展。该动作条件接口使失败轨迹也能用于监督动作后果，将错误动作转化为有效的未来目标而非被丢弃。失败感知训练使进展预测器同时学习成功与失败的结果，可在推理时用于评分候选动作。在仿真和真实世界双臂操作任务上的大量实验表明，FACT优于多个现有基线，且随着失败数据加入训练性能持续提升，有效缓解了坏动作下的成功偏差未来幻觉问题。
原文摘要 · Abstract (English)
Recent world-action models (WAMs) show that co-training policies with future prediction can provide physical priors for action generation. Building on the future-prediction ability of video models, many WAMs generate future videos and recover actions with inverse-dynamics models, or use these predicted videos as goal conditions for action generation. In both cases, the world model is trained mostly on successful demonstrations and has little reason to predict the consequences of bad actions. We introduce FACT, a causal World-Action Model that predicts future video and task progress conditioned on the executed action. This action-conditioned interface allows failure rollouts to supervise action consequences, turning bad actions into valid future targets rather than being discarded. Failure-aware training makes the progress predictor aware of both successful and failed action outcomes, which can optionally be used to score sampled action candidates at inference. Extensive experiments on simulation and real-world bimanual manipulation tasks show that FACT outperforms many existing baselines, improves as failure data are incorporated into training, and reduces success-biased future hallucination under bad actions. See more details at https://fact-wam.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。