零样本训练让机器人精准抓取变形物体，无需真实数据微调
SimWeaver: Zero-Shot RGB Sim-to-Real for Deformable Manipulation
- 用仿真生成图像和轨迹，构建可扩展的虚拟训练环境
- 5类变形任务平均成功率91%，部分达100%且抗视觉变化
- 适合想低成本部署真实机器人抓取系统的研究者与工程师
针对无真实世界微调的变形物体操作任务，现有方法仍难以解决RGB域外泛化问题。本文提出SimWeaver，仅需每任务200条仿真演示，即可训练出零样本的RGB视觉-语言-动作（VLA）策略，在5种不同变形任务（如塑料袋抓取）中实现单任务成功率超80%、平均成功率91%。该方法融合可靠测量驱动的仿真器（SimWeaver-Sim）、支持单图生成的可扩展资产框架（SimWeaver-Asset）、确定性拓扑感知轨迹合成器（SimWeaver-Syn），以及考虑ISP的光照增强模拟真实分布偏移的模拟到现实协议（SimWeaver-Real）。在丝绸抓取任务中，仿真训练策略在视觉分布偏移下达到100%成功率，而真实数据基线降至9%-70%，且单位轨迹成本低两个数量级。项目代码与代表性资产将公开发布。
原文摘要 · Abstract (English)
RGB sim-to-real for deformable manipulation has remained largely unsolved without real-world fine-tuning. We present SimWeaver, which trains zero-shot RGB VLA policies on 200 simulated demonstrations per task, reaching above 80% per-task and 91% average real-world success across 5 diverse deformable tasks including plastic-bag manipulation, without teleoperation or per-task calibration. SimWeaver combines a reliable measurement-backed simulator (SimWeaver-Sim) with an extensible asset framework supporting single-image generation(SimWeaver-Asset), a deterministic topology-aware trajectory synthesizer (SimWeaver-Syn), and a sim-to-real protocol with ISP-aware photometric augmentation (SimWeaver-Real). On silk grasping, the sim-trained policy reaches 100% under visual distribution shifts where real-data baselines drop to 9-70%, at two orders of magnitude lower per-trajectory cost. We will release SimWeaver and a representative asset subset. Project page: https://simweaver.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。