arXiv:2603.13528cs.ROcs.CV2026-03被引 3

通过虚拟故障生成,让机器人学会自动修复操作失误。

Learning Actionable Manipulation Recovery via Counterfactual Failure Synthesis

  • 在真实演示基础上,用生成模型合成逼真的失败动作序列。
  • 实测纠错准确率从19.7%提升至81.3%,实现零样本闭环恢复。
  • 适合需要自主纠错的工业机器人、服务机器人场景。

尽管基础模型显著提升了机器人操作能力,但系统仍难以自主应对执行错误。现有故障学习方法依赖昂贵且危险的真实数据采集或仿真扰动,带来严重的仿真到现实差距。此外,现有视觉分析器多输出粗粒度的二元诊断,而非可执行的轨迹级修正。为弥合诊断与可行动修复之间的鸿沟,我们提出Dream2Fix框架,直接从成功的实际演示中合成逼真的反事实失败轨迹。通过在生成世界模型中扰动动作,Dream2Fix无需依赖模拟器即可生成成对的故障-修正数据。为确保生成数据对机器人学习具有物理可行性,我们设计了结构化验证机制,严格筛选任务有效性、视觉一致性与运动安全性的轨迹。该引擎生成了超过12万对高保真样本数据集。基于此数据集,我们微调一个视觉语言模型,联合预测故障类型与精确恢复轨迹,将视觉异常直接映射为纠正动作。大量真实机器人实验表明,本方法在纠错准确率上达到业界领先水平,从19.7%提升至81.3%,并在物理部署中成功实现零样本闭环故障恢复。

原文摘要 · Abstract (English)

While recent foundation models have significantly advanced robotic manipulation, these systems still struggle to autonomously recover from execution errors. Current failure-learning paradigms rely on either costly and unsafe real-world data collection or simulator-based perturbations, which introduce a severe sim-to-real gap. Furthermore, existing visual analyzers predominantly output coarse, binary diagnoses rather than the executable, trajectory-level corrections required for actual recovery. To bridge the gap between failure diagnosis and actionable recovery, we introduce Dream2Fix, a framework that synthesizes photorealistic, counterfactual failure rollouts directly from successful real-world demonstrations. By perturbing actions within a generative world model, Dream2Fix creates paired failure-correction data without relying on simulators. To ensure the generated data is physically viable for robot learning, we implement a structured verification mechanism that strictly filters rollouts for task validity, visual coherence, and kinematic safety. This engine produces a high-fidelity dataset of over 120k paired samples. Using this dataset, we fine-tune a vision-language model to jointly predict failure types and precise recovery trajectories, mapping visual anomalies directly to corrective actions. Extensive real-world robotic experiments show our approach achieves state-of-the-art correction accuracy, improving from 19.7% to 81.3% over prior baselines, and successfully enables zero-shot closed-loop failure recovery in physical deployments.

机器人故障恢复生成模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。