用场景与多人交互信息预测10秒长时序人体动作，更真实多样。
Massively Multi-Person 3D Human Motion Forecasting with Scene Context
- 结合时空卷积与Transformer，融合动作与场景信息。
- 在Humans in Kitchens数据集上，生成动作更真实且多样性更高。
- 适合研究人机交互、智能机器人动作预测的开发者。
长期3D人体动作预测极具挑战性：人类行为的随机性使得仅凭输入序列难以生成逼真动作。场景环境及周围人的运动信息能显著提升生成效果。本文提出一种场景感知的社会化Transformer模型（SAST），用于预测长达10秒的人体运动。不同于以往模型，该方法可建模场景中人数和物体数量差异极大的交互关系。我们采用时间卷积编码器-解码器结构，搭配基于Transformer的瓶颈层，高效融合运动与场景信息，并使用去噪扩散模型建模条件动作分布。在包含1至16人、29至50个可见物体的Humans in Kitchens数据集上进行评估，结果表明，本模型在多个指标及用户调研中均优于现有方法，生成动作更具真实性和多样性。代码已公开于https://github.com/felixbmuller/SAST。
原文摘要 · Abstract (English)
Forecasting long-term 3D human motion is challenging: the stochasticity of human behavior makes it hard to generate realistic human motion from the input sequence alone. Information on the scene environment and the motion of nearby people can greatly aid the generation process. We propose a scene-aware social transformer model (SAST) to forecast long-term (10s) human motion motion. Unlike previous models, our approach can model interactions between both widely varying numbers of people and objects in a scene. We combine a temporal convolutional encoder-decoder architecture with a Transformer-based bottleneck that allows us to efficiently combine motion and scene information. We model the conditional motion distribution using denoising diffusion models. We benchmark our approach on the Humans in Kitchens dataset, which contains 1 to 16 persons and 29 to 50 objects that are visible simultaneously. Our model outperforms other approaches in terms of realism and diversity on different metrics and in a user study. Code is available at https://github.com/felixbmuller/SAST.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。