用动作捕捉数据生成雷达微多普勒图，省时省力。
MoCap2Radar: A Spatiotemporal Transformer for Synthesizing Micro-Doppler Radar Signatures from Motion Capture
- 用时空变换器建模人体动作与雷达信号的映射关系。
- 生成的雷达图在视觉和量化指标上都表现良好。
- 适合边缘计算与物联网雷达，可扩充稀缺雷达数据。
我们提出一种纯机器学习方法，从动作捕捉（MoCap）数据合成雷达频谱图。将摩卡到频谱的转换建模为基于Transformer的滑动窗口序列到序列任务，同时捕捉标记点间的空间关系和帧间的时间动态。真实世界实验表明，该方法生成的微多普勒雷达频谱图在视觉和定量评估上均合理，并具备良好的泛化能力。消融实验显示，模型既具备将多部件运动转化为多普勒特征的能力，也理解人体各部位间的空间关系。结果展示了变压器在时间序列信号处理中的潜力，特别适用于边缘计算和物联网雷达。此外，该方法能利用丰富的动作捕捉数据增强稀缺雷达数据集，用于训练高级应用。相比物理建模方法，其计算开销显著更低。
原文摘要 · Abstract (English)
We present a pure machine learning process for synthesizing radar spectrograms from Motion-Capture (MoCap) data. We formulate MoCap-to-spectrogram translation as a windowed sequence-to-sequence task using a transformer-based model that jointly captures spatial relations among MoCap markers and temporal dynamics across frames. Real-world experiments show that the proposed approach produces visually and quantitatively plausible doppler radar spectrograms and achieves good generalizability. Ablation experiments show that the learned model includes both the ability to convert multi-part motion into doppler signatures and an understanding of the spatial relations between different parts of the human body. The result is an interesting example of using transformers for time-series signal processing. It is especially applicable to edge computing and Internet of Things (IoT) radars. It also suggests the ability to augment scarce radar datasets using more abundant MoCap data for training higher-level applications. Finally, it requires far less computation than physics-based methods for generating radar data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。