构建首个融合毫米波雷达与RGB-D的多传感器数据集，支持复杂环境感知研究。
RadarRGBD A Multi-Sensor Fusion Dataset for Perception with RGB-D and mmWave Radar
- 融合毫米波雷达点云与RGB-D数据，支持室内外及低光场景。
- 采用高分辨率雷达与原始数据，提升传感器融合研究基础。
- 改进深度估计方法，有效修复遮挡导致的深度缺失问题。
多传感器融合在室内外环境感知中具有巨大潜力，尤其在恶劣天气和低光照条件下，毫米波雷达与RGB-D传感器的结合展现出显著优势。然而，现有自动驾驶与机器人领域的多传感器数据集普遍缺乏高质量毫米波雷达数据。为此，我们提出了全新的多传感器数据集RadarRGBD，包含RGB-D数据、毫米波雷达点云及原始雷达矩阵，覆盖多种室内外场景及低光照环境。相比现有数据集，RadarRGBD采用更高分辨率毫米波雷达并提供原始数据，为毫米波雷达与视觉传感器融合研究提供了新基础。此外，针对Kinect V2因遮挡和匹配误差导致的深度图噪声与空洞问题，我们微调了一个开源相对深度估计框架，并引入数据集中绝对深度信息进行监督，同时加入伪相对深度尺度信息以优化全局深度尺度估计。实验结果表明，该方法能有效填补传感器数据中的缺失区域。数据集及相关文档将公开发布于：https://github.com/song4399/RadarRGBD。
原文摘要 · Abstract (English)
Multi-sensor fusion has significant potential in perception tasks for both indoor and outdoor environments. Especially under challenging conditions such as adverse weather and low-light environments, the combined use of millimeter-wave radar and RGB-D sensors has shown distinct advantages. However, existing multi-sensor datasets in the fields of autonomous driving and robotics often lack high-quality millimeter-wave radar data. To address this gap, we present a new multi-sensor dataset:RadarRGBD. This dataset includes RGB-D data, millimeter-wave radar point clouds, and raw radar matrices, covering various indoor and outdoor scenes, as well as low-light environments. Compared to existing datasets, RadarRGBD employs higher-resolution millimeter-wave radar and provides raw data, offering a new research foundation for the fusion of millimeter-wave radar and visual sensors. Furthermore, to tackle the noise and gaps in depth maps captured by Kinect V2 due to occlusions and mismatches, we fine-tune an open-source relative depth estimation framework, incorporating the absolute depth information from the dataset for depth supervision. We also introduce pseudo-relative depth scale information to further optimize the global depth scale estimation. Experimental results demonstrate that the proposed method effectively fills in missing regions in sensor data. Our dataset and related documentation will be publicly available at: https://github.com/song4399/RadarRGBD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。