用双分支网络提升透明物体深度图补全精度。
TDCNet: Transparent Objects Depth Completion with CNN-Transformer Dual-Branch Parallel Network
- CNN与Transformer并行处理深度图和RGB-D数据
- 在多个公开数据集上达到当前最优性能
- 适合做机器人感知与三维重建的研究者
透明物体的感知与操作是工业和实验室机器人中的关键挑战。传统传感器因表面折射、反射及缺乏可见纹理，难以获取完整的深度信息。以往研究尝试通过深度学习模型，从RGB图像和受损深度图中恢复完整深度图，但未能充分挖掘原始深度图的信息，导致补全精度受限。为此，我们提出TDCNet，一种基于卷积神经网络（CNN）与视觉变压器（Transformer）双分支并行结构的透明物体深度补全方法。其中一个分支提取部分深度图特征，另一分支处理RGB-D图像。实验结果表明，该模型在多个公开数据集上均取得当前最优表现。代码与预训练模型已开源：https://github.com/XianghuiFan/TDCNet。
原文摘要 · Abstract (English)
The sensing and manipulation of transparent objects present a critical challenge in industrial and laboratory robotics. Conventional sensors face challenges in obtaining the full depth of transparent objects due to the refraction and reflection of light on their surfaces and their lack of visible texture. Previous research has attempted to obtain complete depth maps of transparent objects from RGB and damaged depth maps (collected by depth sensor) using deep learning models. However, existing methods fail to fully utilize the original depth map, resulting in limited accuracy for deep completion. To solve this problem, we propose TDCNet, a novel dual-branch CNN-Transformer parallel network for transparent object depth completion. The proposed framework consists of two different branches: one extracts features from partial depth maps, while the other processes RGB-D images. Experimental results demonstrate that our model achieves state-of-the-art performance across multiple public datasets. Our code and the pre-trained model are publicly available at https://github.com/XianghuiFan/TDCNet.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。