用3D重建和物理仿真生成特定场景毫米波雷达数据，无需实测即可训练感知模型。
RadarTwin: Scene-Specific mmWave Radar Simulation and Learning for Mobile Indoor Perception
- 基于3D重建与视觉语言模型，用物理射线追踪生成含多路径的雷达原始信号。
- 纯仿真训练模型在12类物体识别中准确率达95.3%，仅需少量真实标签。
- 适用于新环境部署前的雷达感知预训练，尤其适合移动室内场景。
毫米波雷达感知受限于数据稀缺：现有数据集训练的模型难以泛化到新物体、新环境和新探测轨迹。我们提出RadarTwin，一个在真实数据采集前生成部署特定雷达训练数据的框架。给定目标空间的3D重建（如手机激光雷达、机器人传感或RGB转3D），RadarTwin利用视觉语言模型推断雷达相关表面材质，并通过物理基射线追踪合成包含多路径传播的原始调频连续波（FMCW）雷达测量值。为研究仿真到现实的迁移效果，我们构建了一个涵盖家庭物品、材质类别、距离、旋转、平移及移动感知轨迹的配对实测-仿真数据集。结果表明，仿真与真实雷达共享相同的目标判别性形状与材质特征，且建模环境多路径至关重要。仅使用仿真数据训练的表示，在无真实雷达标签情况下，对真实物体的识别准确率提升至2.5倍；加入少量标注样本后，12类识别任务准确率达95.3%。RadarTwin实现了在新空间中无需任何真实雷达数据即可训练感知模型。
原文摘要 · Abstract (English)
Millimeter-wave (mmWave) radar perception is limited by data scarcity: models trained on existing radar datasets fail to generalize to new objects, environments, and sensing trajectories. We present RadarTwin, a framework for generating deployment-specific radar training data before real data collection. Given a 3D reconstruction of a target space (phone LiDAR, robot-mounted sensing, or RGB-to-3D), RadarTwin uses a vision-language model to infer radar-relevant surface materials and a physics-based ray tracer to synthesize raw frequency-modulated continuous-wave (FMCW) radar measurements with multi-bounce propagation. To study what transfers from simulation to reality, we collect a paired real-simulated dataset spanning household objects, material classes, distances, rotations, translations, and mobile sensing trajectories. We show that simulated and real radar share the same object-discriminative shape and material features, and that modeling the environment's multipath is essential to matching real measurements. A representation trained on simulation alone recognizes real objects at 2.5 times chance with no real radar labels, and a few labeled examples raise this to 95.3% on a 12-way recognition task. RadarTwin enables training radar perception for a new space before any real radar data is collected there.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。