构建66万帧毫米波雷达人体重建数据集，支持隐私保护的三维动作捕捉。
M4Human: A Large-Scale Multimodal mmWave Radar Benchmark for Human Mesh Reconstruction
- 融合毫米波雷达、RGB与深度数据，提供原始张量和点云两种信号格式。
- 包含20人50种动作，共66.1万帧，规模为此前最大数据集的9倍。
- 适用于雷达人体建模、多模态融合研究，尤其适合运动分析与康复应用。
人体网格重建（HMR）可直接揭示身体与环境的交互，支撑多种沉浸式应用。现有大规模HMR数据集依赖可见光RGB输入，但视觉感知受限于遮挡、光照变化及隐私问题。为克服这些局限，近期研究探索了射频毫米波雷达在隐私保护的室内人体感知中的应用。然而，当前雷达数据集存在骨骼标注稀疏、规模有限、动作简单等问题。为此，我们推出M4Human，目前规模最大（661,000帧）的多模态毫米波雷达基准数据集，是此前最大数据集的9倍。该数据集包含高分辨率毫米波雷达、RGB与深度数据，提供原始雷达张量（RT）与处理后的雷达点云（RPC），支持不同粒度的射频信号研究。数据集包含高质量动作捕捉（MoCap）标注，涵盖3D网格与全局轨迹，覆盖20名受试者和50种多样动作，包括原地、坐姿及自由空间体育或康复动作。我们在RT与RPC模态上建立基准，并开展与RGB-D模态的多模态融合实验。大量结果表明，M4Human对雷达驱动的人体建模具有重要意义，但在快速、非约束运动下仍存挑战。数据集与代码将在论文发表后公开。
原文摘要 · Abstract (English)
Human mesh reconstruction (HMR) provides direct insights into body-environment interaction, which enables various immersive applications. While existing large-scale HMR datasets rely heavily on line-of-sight RGB input, vision-based sensing is limited by occlusion, lighting variation, and privacy concerns. To overcome these limitations, recent efforts have explored radio-frequency (RF) mmWave radar for privacy-preserving indoor human sensing. However, current radar datasets are constrained by sparse skeleton labels, limited scale, and simple in-place actions. To advance the HMR research community, we introduce M4Human, the current largest-scale (661K-frame) ($9\times$ prior largest) multimodal benchmark, featuring high-resolution mmWave radar, RGB, and depth data. M4Human provides both raw radar tensors (RT) and processed radar point clouds (RPC) to enable research across different levels of RF signal granularity. M4Human includes high-quality motion capture (MoCap) annotations with 3D meshes and global trajectories, and spans 20 subjects and 50 diverse actions, including in-place, sit-in-place, and free-space sports or rehabilitation movements. We establish benchmarks on both RT and RPC modalities, as well as multimodal fusion with RGB-D modalities. Extensive results highlight the significance of M4Human for radar-based human modeling while revealing persistent challenges under fast, unconstrained motion. The dataset and code will be released after the paper publication.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。