利用光线折射中间表示,仅用彩色图像实现透明物体6D姿态精准估计。
ReFlow6D: Refraction-Guided Transparent Object 6D Pose Estimation via Intermediate Representation Learning
- 基于光线折射构建与环境无关的中间特征表示。
- 在TOD和Trans32K-6D数据集上显著超越现有方法。
- 适合需要高精度透明物体感知的机器人抓取场景。
透明物体在日常生活中普遍存在,其感知与机器人操作至关重要。然而,由于其独特的折射与反射特性,准确估计6D姿态面临巨大挑战。为此,我们提出ReFlow6D,一种利用折射中间表示的新型透明物体6D姿态估计方法。不同于传统方法,该方法在不受RGB图像空间变化影响且独立于深度信息的特征空间中建模。受图像抠图启发,我们模拟光线穿过透明物体时的路径变形,生成由光折射引导的、具有物体特性的中间表示,该表示与观测环境无关。通过将这些中间特征融入姿态估计网络,ReFlow6D仅需输入RGB图像即可实现精确的6D姿态估计。此外,我们引入一种新的透明物体合成损失,促进更优折射中间特征的生成。实验证明,该方法在TOD和Trans32K-6D数据集上显著优于当前最优方法。机器人抓取实验进一步表明,其姿态估计精度可有效转化为实际任务性能。源代码已开源:https://github.com/StoicGilgamesh/ReFlow6D 和 https://github.com/StoicGilgamesh/matting_rendering。
原文摘要 · Abstract (English)
Transparent objects are ubiquitous in daily life, making their perception and robotics manipulation important. However, they present a major challenge due to their distinct refractive and reflective properties when it comes to accurately estimating the 6D pose. To solve this, we present ReFlow6D, a novel method for transparent object 6D pose estimation that harnesses the refractive-intermediate representation. Unlike conventional approaches, our method leverages a feature space impervious to changes in RGB image space and independent of depth information. Drawing inspiration from image matting, we model the deformation of the light path through transparent objects, yielding a unique object-specific intermediate representation guided by light refraction that is independent of the environment in which objects are observed. By integrating these intermediate features into the pose estimation network, we show that ReFlow6D achieves precise 6D pose estimation of transparent objects, using only RGB images as input. Our method further introduces a novel transparent object compositing loss, fostering the generation of superior refractive-intermediate features. Empirical evaluations show that our approach significantly outperforms state-of-the-art methods on TOD and Trans32K-6D datasets. Robot grasping experiments further demonstrate that ReFlow6D's pose estimation accuracy effectively translates to real-world robotics task. The source code is available at: https://github.com/StoicGilgamesh/ReFlow6D and https://github.com/StoicGilgamesh/matting_rendering.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。