构建首个毫米波非视距感知数据集，助力物体穿透遮挡识别
MITO: A Millimeter-Wave Dataset and Simulator for Non-Line-of-Sight Perception
- 用机械臂移动雷达采集多频毫米波信号，合成高分辨率图像
- 生成550张高分辨毫米波图像，覆盖76种日常物体的可视与非可视场景
- 开源模拟器支持任意3D模型生成毫米波图像，适合机器人感知研究
感知世界是推理与决策的基础，但视觉常因遮挡失效。我们提出MITO，首个包含多样化日常物体的毫米波（mmWave）数据集，通过UR5机械臂操控双频mmWave雷达和RGB-D相机采集。相比可见光，毫米波可穿透纸箱、布料、塑料等常见遮挡物，但单帧分辨率低。为此，利用机械臂运动实现合成孔径，融合多帧提升图像质量。数据集包含超过2400万帧毫米波数据，生成550张高分辨率合成孔径毫米波图像（含视线内与非视线内），并配套提供RGB-D图像、分割掩码及原始信号。我们还开发了开源仿真工具，可为任意3D三角网格生成合成毫米波图像。最后，基于该数据集建立非视距分割与分类基准，验证其在扩展非视距感知中的实用性。
原文摘要 · Abstract (English)
The ability to observe the world is fundamental to reasoning and making informed decisions on how to interact with the environment. However, optical perception can often be disrupted due to common occurrences, such as occlusions, which can pose challenges to existing vision systems. We present MITO, the first millimeter-wave (mmWave) dataset of diverse, everyday objects, collected using a UR5 robotic arm with two mmWave radars operating at different frequencies and an RGB-D camera. Unlike visible light, mmWave signals can penetrate common occlusions (e.g., cardboard boxes, fabric, plastic) but each mmWave frame has much lower resolution than typical cameras. To capture higher-resolution mmWave images, we leverage the robot's mobility and fuse frames over the synthesized aperture. MITO captures over 24 million mmWave frames and uses them to generate 550 high-resolution mmWave (synthetic aperture) images in line-of-sight and non-light-of-sight (NLOS), as well as RGB-D images, segmentation masks, and raw mmWave signals, taken from 76 different objects. We develop an open-source simulation tool that can be used to generate synthetic mmWave images for any 3D triangle mesh. Finally, we demonstrate the utility of our dataset and simulator for enabling broader NLOS perception by developing benchmarks for NLOS segmentation and classification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。