用变压器模型从互动视频中学习两人互动生成动作，提升动作一致性。
Learning Reactive Human Motion Generation from Paired Interaction Data Using Transformer-Based Models
- 基于配对动作数据，用Transformer模型建模两人互动中的相互影响。
- 简单Transformer生成动作自然，未出现姿态崩溃；iTransformer和Crossformer随时间误差累积。
- 引入人物ID嵌入，有效防止结构坍塌，适合动作捕捉与虚拟人交互应用。
深度学习的发展使文本生成视频和视频预测成为可能。在人体动作建模中，也可根据文本或单人动作序列生成或预测动作。然而这些方法多关注单主体动作生成。本文研究在互动场景中，基于一人动作生成另一人动作的问题，此时两人的动作相互依赖。我们从拳击比赛视频中构建了一个包含配对动作-反应序列的数据集，并探究了基于Transformer模型在此任务上的有效性。具体实现并比较了三种模型：简单Transformer、iTransformer和Crossformer。此外，引入人物ID嵌入以显式区分个体，帮助模型保持结构一致性并更好捕捉互动动态。实验表明，简单Transformer能生成合理且具备交互感知的动作，未出现姿态崩溃；而iTransformer和Crossformer随时间积累误差，导致动作不稳定。同时，所提人物ID嵌入有助于防止结构崩溃，提升动作一致性。结果凸显了在交互感知动作生成中显式建模个体身份的重要性。
原文摘要 · Abstract (English)
Recent advances in deep learning have enabled the generation of videos from textual descriptions as well as the prediction of future sequences from input videos. Similarly, in human motion modeling, motions can be generated from text or predicted from a single person's motion sequence. However, these approaches primarily focus on single-agent motion generation. In contrast, this study addresses the problem of generating the motion of one person based on the motion of another in interaction scenarios, where the two motions are mutually dependent. We construct a dataset of paired action-reaction motion sequences extracted from boxing match videos and investigate the effectiveness of Transformer-based models for this task. Specifically, we implement and compare three models: a simple Transformer, iTransformer, and Crossformer. In addition, we introduce a person ID embedding to explicitly distinguish between individuals, enabling the model to maintain structural consistency and better capture interaction dynamics. Experimental results show that the simple Transformer can generate plausible interaction-aware motions without suffering from posture collapse, while iTransformer and Crossformer accumulate errors over time, leading to unstable motion generation. Furthermore, the proposed person ID embedding contributes to preventing structural collapse and improving motion consistency. These results highlight the importance of explicitly modeling individual identity in interaction-aware motion generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。