单张图像实现透明物体分割与深度估计，精度显著提升。
Monocular Depth Estimation and Segmentation for Transparent Object with Iterative Semantic and Geometric Fusion
- 设计迭代式语义与几何融合模块，跨任务整合多尺度信息。
- 在合成与真实数据集上，性能超越主流方法38.8%~46.2%。
- 仅需单张RGB图像，适合实际机器人场景部署。
透明物体感知对众多机器人任务至关重要。然而，由于复杂的光学特性，准确分割和估计透明物体的深度仍具挑战性。现有方法通常仅聚焦单一任务，依赖额外输入或专用传感器，忽视任务间的交互与后续优化过程，导致预测结果模糊且不理想。为此，我们提出首个仅用单张图像即可在透明物体分割与深度估计两任务上均表现优异的单目框架。具体而言，设计了一种新型语义与几何融合模块，有效整合任务间多尺度信息；并借鉴人类感知机制，引入迭代策略，逐步优化初始特征以获得更清晰结果。在两个具有挑战性的合成与真实世界数据集上的实验表明，该模型在仅使用单个RGB输入的情况下，相比最先进的单目、立体及多视角方法，性能提升约38.8%至46.2%。代码与模型已公开于https://github.com/L-J-Yuan/MODEST。
原文摘要 · Abstract (English)
Transparent object perception is indispensable for numerous robotic tasks. However, accurately segmenting and estimating the depth of transparent objects remain challenging due to complex optical properties. Existing methods primarily delve into only one task using extra inputs or specialized sensors, neglecting the valuable interactions among tasks and the subsequent refinement process, leading to suboptimal and blurry predictions. To address these issues, we propose a monocular framework, which is the first to excel in both segmentation and depth estimation of transparent objects, with only a single-image input. Specifically, we devise a novel semantic and geometric fusion module, effectively integrating the multi-scale information between tasks. In addition, drawing inspiration from human perception of objects, we further incorporate an iterative strategy, which progressively refines initial features for clearer results. Experiments on two challenging synthetic and real-world datasets demonstrate that our model surpasses state-of-the-art monocular, stereo, and multi-view methods by a large margin of about 38.8%-46.2% with only a single RGB input. Codes and models are publicly available at https://github.com/L-J-Yuan/MODEST.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。