用仿真数据训练机械臂抓取传送带动态物体，真实场景成功率超97%。
Sim-to-Real Dynamic Object Manipulation on Conveyor Systems via Optimization Path Shaping
- 通过模拟生成数据并优化训练路径，提升模型对几何特征的感知能力
- 无需真实测试数据，在真实餐厅场景中完成超1万次操作，成功率>97%
- 适用于不同背景、机器人形态和物体形状，泛化能力强
在传送带上实现可泛化的动态物体操作对提升制造效率至关重要，因其可避免为不同场景定制工程方案。为此，模仿学习凭借专家示范成为有前景的方法，但其泛化能力受限于示范数据量，而真实动态物体操作数据稀缺且采集耗时。本文通过在仿真环境中生成示范数据解决这一问题。模拟数据与真实观测间的外观差异是主要挑战，为此提出几何增强模型（GEM），采用设计的外观噪声渐进策略，引导优化路径，优先关注观测中的几何信息。大量仿真实验与真实任务验证表明，GEM可在环境背景、机器人形态、运动动态及物体几何间实现良好泛化。特别地，该方法已部署于真实餐厅场景进行餐具收集，未使用任何测试场景数据，成功完成超过10,000次操作，成功率超过97%。
原文摘要 · Abstract (English)
Realizing generalizable dynamic object manipulation on conveyor systems is important for enhancing manufacturing efficiency, as it eliminates specialized engineering for different scenarios. To this end, imitation learning emerges as a promising paradigm, leveraging expert demonstrations to teach a policy manipulation skills. Although the generalization of an imitation learning policy can be improved by increasing demonstrations, demonstration collection is labor-intensive. Besides, public dynamic object manipulation data is scarce. In this work, we address this data scarcity problem via generating demonstrations in a simulator. A significant challenge of using simulated data lies in the appearance gap between simulated and real-world observations. To tackle this challenge, we propose Geometry-Enhanced Model (GEM), which employs our designed appearance noise annealing strategy to shape the policy optimization path, thereby prioritizing the geometry information in observations. Extensive experiments in simulated and real-world tasks demonstrate that GEM can generalize across environment backgrounds, robot embodiments, motion dynamics, and object geometries. Notably, GEM is deployed in a real canteen for tableware collection. Without test-scene data, GEM achieves a success rate of over 97% across more than 10,000 operations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。