arXiv:2603.20669cs.ROcs.CV2026-03

解决短距ToF相机在大场景下的深度补全问题,实现轻量化实时部署。

ToFormer: Towards Large-scale Scenario Depth Completion for Lightweight ToF Camera

  • 构建新型3D-2D融合网络,利用联合传播池化与多模态注意力建模长程关系。
  • 在LASER-ToF数据集上误差降低8.6%,优于现有最佳方法。
  • 可部署于无人机平台,支持10Hz实时运行,适用于复杂环境大场景导航。

飞行时间(ToF)相机因结构紧凑、精度高,广泛应用于机器人任务中。然而其感知范围有限,难以在大场景中使用。深度补全被视作扩展其感知范围的可行方案,但现有研究缺乏专用数据集且难以泛化至真实ToF测量。本文提出一个全栈式框架,实现短距ToF相机在大场景中的深度补全。首先,搭建多传感器平台并基于重建流程采集真实世界中的密集大尺度真值数据,构建首个大规模场景ToF深度补全数据集LASER-ToF。其次,提出一种感知传感器特性的深度补全网络,包含新颖的3D分支与3D-2D联合传播池化(JPP)模块及多模态交叉协方差注意力(MXCA),有效建模长程依赖并高效融合3D-2D信息,尤其在非均匀深度稀疏条件下表现优异;同时可利用视觉SLAM生成的稀疏点云作为补充,进一步提升预测精度。实验表明,本方法均方误差比第二佳方法低8.6%,且保持轻量化设计,支持机载部署。最后,在四旋翼无人机上以10Hz运行验证系统实用性,成功实现复杂环境下可靠的大规模建图与远距离规划。

原文摘要 · Abstract (English)

Time-of-Flight (ToF) cameras possess compact design and high measurement precision to be applied to various robot tasks. However, their limited sensing range restricts deployment in large-scale scenarios. Depth completion has emerged as a potential solution to expand the sensing range of ToF cameras, but existing research lacks dedicated datasets and struggles to generalize to ToF measurements. In this paper, we propose a full-stack framework that enables depth completion in large-scale scenarios for short-range ToF cameras. First, we construct a multi-sensor platform with a reconstruction-based pipeline to collect real-world ToF samples with dense large-scale ground truth, yielding the first LArge-ScalE scenaRio ToF depth completion dataset (LASER-ToF). Second, we propose a sensor-aware depth completion network that incorporates a novel 3D branch with a 3D-2D Joint Propagation Pooling (JPP) module and Multimodal Cross-Covariance Attention (MXCA), enabling effective modeling of long-range relationships and efficient 3D-2D fusion under non-uniform ToF depth sparsity. Moreover, our network can utilize the sparse point cloud from visual SLAM as a supplement to ToF depth to further improve prediction accuracy. Experiments show that our method achieves an 8.6% lower mean absolute error than the second-best method, while maintaining lightweight design to support onboard deployment. Finally, to verify the system's applicability on real robots, we deploy proposed method on a quadrotor at a 10Hz runtime, enabling reliable large-scale mapping and long-range planning in challenging environments for short-range ToF cameras.

深度补全ToF相机轻量化无人机

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。