arXiv:2511.09022eess.SPcs.CV2025-11被引 1

用视频生成雷达多普勒图,模拟人体活动识别场景。

RadHARSimulator V2: Video to Doppler Generator

  • 从视频直接生成多普勒谱,融合视觉与雷达建模。
  • 实现自由空间与穿墙场景的回波仿真,生成高精度多普勒时间图。
  • 适合雷达感知、智能监控与数据合成研究者使用。

基于雷达的人体活动识别(HAR)仍缺乏全面的仿真方法。现有软件依赖模型或动作捕捉数据,灵活性不足。本文提出一种新模拟器(RadHARSimulator V2),可直接从视频生成多普勒谱。该系统包含计算机视觉与雷达模块:视觉部分采用实时目标检测与全局最近邻跟踪,通过高分辨率网络估计二维姿态,再以最近邻匹配法获取三维姿态,并用卡尔曼滤波实现平滑时序三维估计。雷达模块中,先用Savitzky-Golay法插值平滑姿态;再结合延迟模型与镜像法模拟自由空间及穿墙场景回波;经脉冲压缩、动目标指示与DnCNN处理生成距离-时间图;再利用短时傅里叶变换与DnCNN生成多普勒-时间图(DTM);最后用最大局部能量法提取DTM上的脊线特征。此外,提出一种混合并行-串行神经网络用于雷达HAR。数值实验验证了模拟器与网络的有效性。开源代码见:https://github.com/JoeyBGOfficial/RadHARSimulatorV2-Video-to-Doppler-Generator。

原文摘要 · Abstract (English)

Radar-based human activity recognition (HAR) still lacks a comprehensive simulation method. Existing software is developed based on models or motion-captured data, resulting in limited flexibility. To address this issue, a simulator that directly generates Doppler spectra from recorded video footage (RadHARSimulator V2) is presented in this paper. Both computer vision and radar modules are included in the simulator. In computer vision module, the real-time model for object detection with global nearest neighbor is first used to detect and track human targets in the video. Then, the high-resolution network is used to estimate two-dimensional poses of the detected human targets. Next, the three-dimensional poses of the detected human targets are obtained by nearest matching method. Finally, smooth temporal three-dimensional pose estimation is achieved through Kalman filtering. In radar module, pose interpolation and smoothing are first achieved through the Savitzky-Golay method. Second, the delay model and the mirror method are used to simulate echoes in both free-space and through-the-wall scenarios. Then, range-time map is generated using pulse compression, moving target indication, and DnCNN. Next, Doppler-time map (DTM) is generated using short-time Fourier transform and DnCNN again. Finally, the ridge features on the DTM are extracted using the maximum local energy method. In addition, a hybrid parallel-serial neural network architecture is proposed for radar-based HAR. Numerical experiments are conducted and analyzed to demonstrate the effectiveness of the designed simulator and the proposed network model. The open-source code of this work can be found in: https://github.com/JoeyBGOfficial/RadHARSimulatorV2-Video-to-Doppler-Generator.

雷达仿真视频生成姿态估计多普勒图

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。