用网格附着辐射场生成真实感服装图像，解决仿真到现实的视觉差距问题。
GARField: Addressing the visual Sim-to-Real gap in garment manipulation with mesh-attached radiance fields
- 基于三角网格模拟状态，通过可微渲染生成逼真图像。
- 首次实现从模拟网格直接生成真实观测数据，无需复杂采集流程。
- 适合研究机器人服装操作与仿真数据增强的学者使用。
人类能快速准确地操控衣物，而机器人却面临巨大挑战。关键原因在于人类能预先想象操作结果，从而建立意图与姿态之间的预测关系，可在遮挡状态下规划、动态调整并应对突发情况。相比之下，机器人难以形成此类直觉，且计划与观察间关联薄弱。这在很大程度上归因于获取高质量、高密度标注的纺织品操作数据成本高昂，无论在质量还是数量上均存在瓶颈。当前，生成高质量带标签的服装操作数据主要依赖复杂的现实世界数据采集流程，仅能获得简化状态估计。本文提出一种新方法：从物体状态生成真实世界观测数据。为此，我们提出GARField（服装附着辐射场），据我们所知是首个用于从三角网格存储的模拟状态中生成数据的可微渲染架构。代码已公开于 https://ddonatien.github.io/garfield-website/
原文摘要 · Abstract (English)
While humans intuitively manipulate garments and other textile items swiftly and accurately, it is a significant challenge for robots. A factor crucial to human performance is the ability to imagine, a priori, the intended result of the manipulation intents and hence develop predictions on the garment pose. That ability allows us to plan from highly obstructed states, adapt our plans as we collect more information and react swiftly to unforeseen circumstances. Conversely, robots struggle to establish such intuitions and form tight links between plans and observations. We can partly attribute this to the high cost of obtaining densely labelled data for textile manipulation, both in quality and quantity. The problem of data collection is a long-standing issue in data-based approaches to garment manipulation. As of today, generating high-quality and labelled garment manipulation data is mainly attempted through advanced data capture procedures that create simplified state estimations from real-world observations. However, this work proposes a novel approach to the problem by generating real-world observations from object states. To achieve this, we present GARField (Garment Attached Radiance Field), the first differentiable rendering architecture, to our knowledge, for data generation from simulated states stored as triangle meshes. Code is available on https://ddonatien.github.io/garfield-website/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。