用低帧率深度传感器实现高帧率稠密深度图，省电又清晰
Depth on Demand: Streaming Dense Depth from a Low Frame Rate Active Sensor
- 融合高帧率摄像头与低帧率深度传感器，分三步完成深度补全
- 在室内室外数据集上实现更高密度的三维重建，帧率提升显著
- 适合机器人导航、自动驾驶等对实时性要求高的场景
高帧率且精确的深度估计在机器人和汽车感知任务中至关重要。目前，这主要依赖于ToF（室内）和LiDAR（室外）设备，但受限于低帧率、高能耗和空间稀疏性。本文提出的Depth on Demand（DoD）方法，通过结合高帧率RGB相机与可能低帧率、稀疏的主动深度传感器，实现了时空稠密深度的重建。该方案通过三个核心阶段——多模态编码、迭代多模态融合、深度解码——显著降低对深度传感器的传输需求，同时实现更低能耗和更稠密的形状重建。我们在室内和室外视频数据集上进行了广泛验证，涵盖环境扫描和自动驾驶感知等多种应用场景。
原文摘要 · Abstract (English)
High frame rate and accurate depth estimation plays an important role in several tasks crucial to robotics and automotive perception. To date, this can be achieved through ToF and LiDAR devices for indoor and outdoor applications, respectively. However, their applicability is limited by low frame rate, energy consumption, and spatial sparsity. Depth on Demand (DoD) allows for accurate temporal and spatial depth densification achieved by exploiting a high frame rate RGB sensor coupled with a potentially lower frame rate and sparse active depth sensor. Our proposal jointly enables lower energy consumption and denser shape reconstruction, by significantly reducing the streaming requirements on the depth sensor thanks to its three core stages: i) multi-modal encoding, ii) iterative multi-modal integration, and iii) depth decoding. We present extended evidence assessing the effectiveness of DoD on indoor and outdoor video datasets, covering both environment scanning and automotive perception use cases.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。