用视觉感知实现可变形物体的零样本真实世界操控
A Distributional Treatment of Real2Sim2Real for Object-Centric Agent Adaptation in Vision-Driven Deformable Linear Object Manipulation
- 通过无似然推断获取物体物理参数后验分布
- 仿真训练策略在真实世界零样本部署成功
- 适合机器人操控与视觉伺服研究者
我们提出一个端到端框架，解决基于视觉感知的可变形线性物体（DLOs）操控中的真实→仿真→真实（Real2Sim2Real）问题。针对一组参数化表示的DLOs，采用无似然推断（LFI）从动态操作轨迹的视觉与本体感觉数据中估计其物理参数的后验分布，进而用于仿真中的领域随机化。在此基础上，使用无模型强化学习训练仅依赖视觉与本体感觉输入的物体特定视觉-运动策略，完成DLO抓取任务。实验表明，该方法可在不进行任何微调的情况下，将仿真训练的策略直接部署于真实世界。进一步评估了主流LFI方法在仅使用动态轨迹数据时对参数化DLO集的细粒度分类能力，并分析了由此产生的领域分布对仿真策略学习与真实表现的影响。
原文摘要 · Abstract (English)
We present an integrated (or end-to-end) framework for the Real2Sim2Real problem of manipulating deformable linear objects (DLOs) based on visual perception. Working with a parameterised set of DLOs, we use likelihood-free inference (LFI) to compute the posterior distributions for the physical parameters using which we can approximately simulate the behaviour of each specific DLO. We use these posteriors for domain randomisation while training, in simulation, object-specific visuomotor policies (i.e. assuming only visual and proprioceptive sensory) for a DLO reaching task, using model-free reinforcement learning. We demonstrate the utility of this approach by deploying sim-trained DLO manipulation policies in the real world in a zero-shot manner, i.e. without any further fine-tuning. In this context, we evaluate the capacity of a prominent LFI method to perform fine classification over the parametric set of DLOs, using only visual and proprioceptive data obtained in a dynamic manipulation trajectory. We then study the implications of the resulting domain distributions in sim-based policy learning and real-world performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。