用仿真器在线优化机器人抓布动作，提升真实场景下的成功率和鲁棒性。
Enabling Robust Cloth Manipulation via Inference-Time Simulator-in-the-Loop Refinement
- 通过可扩展的合成数据生成与推理时回滚流程，实现布料状态精准建模。
- 在真实机器人上实测成功率达82.3%，显著优于基线方法。
- 适合需要高鲁棒性的智能抓取与柔性物体操控任务。
仿真器-闭环优化为机器人操作提供了一种有前景的推理时机制。它利用物理仿真器作为后端滚动引擎，对候选轨迹进行并行评估并在线优化初始动作，该范式在刚体操作中已被证明有效，但其在从单张RGB图像进行真实世界布料操作中的应用仍具挑战。本文通过三大支柱实现该目标：(i) 设计基于FLASH（一种兼顾物理保真度、数值稳定性和回滚效率的可变形物体仿真器）的可扩展合成数据生成与推理时回滚管道；(ii) 开发纯合成数据训练的现实到仿真模块，将单张RGB观测映射为仿真兼容的布料状态，融合预训练视觉特征与可学习规范令牌；(iii) 通过稀疏网格回滚后端与先验引导的MPPI结合，在离线提炼策略轨迹基础上实现在线规划，保持操作相关的形变与接触特性的同时支持充足的并行回滚批次。真实机器人实验表明，该方法相比基线方法具有更高的成功率和更强的鲁棒性。
原文摘要 · Abstract (English)
Simulator-in-the-loop optimization offers a promising inference-time mechanism for robot manipulation. It uses a physical simulator as a backend rollout engine to evaluate candidate trajectories in parallel and refine nominal actions online, a paradigm proven effective in rigid-body manipulation where state and contact are relatively tractable. We bring this paradigm to real-world cloth manipulation from a single RGB input through three pillars. (i) We design a scalable synthetic-data generation and inference-time rollout pipeline built on FLASH, a deformable-object simulator that provides a practical balance among physical fidelity, numerical stability, and rollout efficiency. (ii) We develop a real-to-sim module, trained purely on synthetic data, that maps a single RGB observation to simulation-compatible cloth state by fusing pretrained visual features with learnable canonical tokens. (iii) We perform online planning by coupling a sparse-mesh rollout backend with prior-guided MPPI, anchored at an offline-distilled policy trajectory, preserving manipulation-relevant deformation and contact while enabling sufficient parallel rollout batches. Real-robot experiments show higher success rates and stronger robustness than baseline methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。