提升低成本dToF传感器在真实场景下的深度图质量。
DEPTHOR++: Robust Depth Enhancement from a Real-World Lightweight dToF and RGB Guidance
- 用合成数据模拟真实噪声,训练更鲁棒的深度补全模型。
- 无参数异常检测机制,有效剔除错误深度测量值。
- 融合RGB与预训练深度先验,显著改善复杂区域恢复效果。
深度增强将原始dToF信号转换为稠密深度图,对高精度任务如3D重建和SLAM至关重要。现有方法多假设理想dToF输入与完美对齐,忽略校准误差与异常值,限制了实际应用。本文系统分析真实轻量级dToF传感器的噪声特性,提出新型鲁棒深度补全框架DEPTHOR++,从三方面提升抗噪能力:首先,基于合成数据集生成真实感训练样本;其次,设计无学习参数的异常检测机制,识别并移除错误dToF测量值,防止误导传播;最后,构建适配噪声dToF输入的深度补全网络,融合RGB图像与预训练单目深度先验,提升困难区域恢复性能。在ZJU-L5和真实数据上,训练策略显著提升现有模型性能,本模型平均降低RMSE 22%、Rel 11%;在Mirror3D-NYU数据集上,镜面区域性能超越前SOTA 37%;在Hammer数据集上,使用RealSense L515模拟低代价dToF数据,平均优于原设备22%,证明低成本传感器可超越高端设备。跨多种真实数据集的定性结果验证了方法的有效性与泛化能力。
原文摘要 · Abstract (English)
Depth enhancement, which converts raw dToF signals into dense depth maps using RGB guidance, is crucial for improving depth perception in high-precision tasks such as 3D reconstruction and SLAM. However, existing methods often assume ideal dToF inputs and perfect dToF-RGB alignment, overlooking calibration errors and anomalies, thus limiting real-world applicability. This work systematically analyzes the noise characteristics of real-world lightweight dToF sensors and proposes a practical and novel depth completion framework, DEPTHOR++, which enhances robustness to noisy dToF inputs from three key aspects. First, we introduce a simulation method based on synthetic datasets to generate realistic training samples for robust model training. Second, we propose a learnable-parameter-free anomaly detection mechanism to identify and remove erroneous dToF measurements, preventing misleading propagation during completion. Third, we design a depth completion network tailored to noisy dToF inputs, which integrates RGB images and pre-trained monocular depth estimation priors to improve depth recovery in challenging regions. On the ZJU-L5 dataset and real-world samples, our training strategy significantly boosts existing depth completion models, with our model achieving state-of-the-art performance, improving RMSE and Rel by 22% and 11% on average. On the Mirror3D-NYU dataset, by incorporating the anomaly detection method, our model improves upon the previous SOTA by 37% in mirror regions. On the Hammer dataset, using simulated low-cost dToF data from RealSense L515, our method surpasses the L515 measurements with an average gain of 22%, demonstrating its potential to enable low-cost sensors to outperform higher-end devices. Qualitative results across diverse real-world datasets further validate the effectiveness and generalizability of our approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。