用视频中可见人体部位生成更鲁棒的动画，提升质量与多样性
Robust Motion Generation using Part-level Reliable Data from Videos
- 将人体分五部分，只用可见部位做训练，避免遮挡干扰
- 在20万条真实动作数据上，生成动作更自然、语义更一致
- 适合做动作生成且关注数据质量的研究者或开发者
从大规模网络视频中提取人类动作，可缓解角色动画中的数据稀缺问题。但许多视频帧中人体部位因出框或遮挡不可见，导致两难：剔除缺失部位的数据会限制规模和多样性，保留则降低数据质量并影响模型性能。为此，我们提出利用视频中清晰可见的局部可信数据，通过一种鲁棒的部件感知掩码自回归模型增强动作生成。首先，将人体分解为五个部分，并识别视频帧中清晰可见的为“可信”部分；其次，使用提出的部件感知变分自编码器将可信部分编码为潜在符号；第三，设计一种鲁棒的部件级掩码生成模型，仅预测被遮挡的可信部分，忽略噪声部分。此外，我们构建了K700-M这一新基准，包含约20万条真实世界动作序列，用于评估。实验结果表明，该方法在清洁与含噪数据集上均优于基线，在动作质量、语义一致性与多样性方面表现更优。
原文摘要 · Abstract (English)
Extracting human motion from large-scale web videos offers a scalable solution to the data scarcity issue in character animation. However, some human parts in many video frames cannot be seen due to off-screen captures or occlusions. It brings a dilemma: discarding the data missing any part limits scale and diversity, while retaining it compromises data quality and model performance. To address this problem, we propose leveraging credible part-level data extracted from videos to enhance motion generation via a robust part-aware masked autoregression model. First, we decompose a human body into five parts and detect the parts clearly seen in a video frame as "credible". Second, the credible parts are encoded into latent tokens by our proposed part-aware variational autoencoder. Third, we propose a robust part-level masked generation model to predict masked credible parts, while ignoring those noisy parts. In addition, we contribute K700-M, a challenging new benchmark comprising approximately 200k real-world motion sequences, for evaluation. Experimental results indicate that our method successfully outperforms baselines on both clean and noisy datasets in terms of motion quality, semantic consistency and diversity. Project page: https://boyuaner.github.io/ropar-main/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。