用RGB和dToF数据提升深度图精度，解决真实场景中的对齐误差问题。
DEPTHOR: Depth Enhancement from a Practical Light-Weight dToF Sensor and RGB Image
- 通过模拟真实dToF噪声训练模型，增强抗干扰能力。
- 融合单目深度估计，改善复杂区域的深度预测效果。
- 在自建数据集上优于现有方法，适合实际部署场景。
深度增强任务利用RGB图像引导，将dToF传感器原始信号转换为高精度、稠密的深度图，是计算机视觉中的关键挑战。现有基于超分辨率的方法在公开数据集上表现良好，但依赖于理想假设，如精确的区域对应关系和可靠的dToF输入，忽略了校准误差导致的错位和异常信号，限制了其在真实场景的应用。为此，我们提出一种新型基于补全的方法DEPTHOR，改进了训练策略与模型架构。首先，我们设计了一种从合成数据集中真实地面真值生成真实dToF数据的模拟方法，实现对噪声鲁棒的训练；其次，设计了一个引入单目深度估计（MDE）的新网络，利用全局深度关系和上下文信息，提升复杂区域的预测性能。在ZJU-L5数据集上，我们的训练策略显著提升了深度补全模型性能，达到与深度超分辨率相当的效果；模型本身实现最新水平，相对误差（Rel）和均方根误差（RMSE）分别降低27%和18%。在自收集的更具挑战性的dToF样本上，相比基于立体匹配的初步地面真值，我们的方法在Rel和RMSE上分别提升23%和22%。代码已开源：https://github.com/ShadowBbBb/Depthor。
原文摘要 · Abstract (English)
Depth enhancement, which uses RGB images as guidance to convert raw signals from dToF into high-precision, dense depth maps, is a critical task in computer vision. Although existing super-resolution-based methods show promising results on public datasets, they often rely on idealized assumptions like accurate region correspondences and reliable dToF inputs, overlooking calibration errors that cause misalignment and anomaly signals inherent to dToF imaging, limiting real-world applicability. To address these challenges, we propose a novel completion-based method, named DEPTHOR, featuring advances in both the training strategy and model architecture. First, we propose a method to simulate real-world dToF data from the accurate ground truth in synthetic datasets to enable noise-robust training. Second, we design a novel network that incorporates monocular depth estimation (MDE), leveraging global depth relationships and contextual information to improve prediction in challenging regions. On the ZJU-L5 dataset, our training strategy significantly enhances depth completion models, achieving results comparable to depth super-resolution methods, while our model achieves state-of-the-art results, improving Rel and RMSE by 27% and 18%, respectively. On a more challenging set of dToF samples we collected, our method outperforms SOTA methods on preliminary stereo-based GT, improving Rel and RMSE by 23% and 22%, respectively. Our Code is available at https://github.com/ShadowBbBb/Depthor
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。