让动作生成模型根据情绪生成自然反应，提升交互真实性。
E-React: Towards Emotionally Controlled Synthesis of Human Reactions
- 用短序列内情绪一致的规律，半监督训练情绪先验
- 在扩散模型中结合空间互动与情绪响应，生成多样化反应
- 适合需要情感化人机交互的场景，如虚拟角色反应合成
情绪是日常人际互动的关键要素。现有动作生成框架忽略情绪影响，导致生成动作不够自然，限制了在交互任务（如人类反应合成）中的应用。本文提出新任务：根据不同情绪提示生成多样化的反应动作。针对有限运动数据下情绪表征学习困难的问题，我们引入一种半监督情绪先验，基于短序列内动作片段情绪一致的观察，训练情绪先验模型。在此基础上，进一步训练一个演员-反应者扩散模型，同时考虑空间交互关系与情绪响应。给定演员的动作序列，该方法可生成在多种情绪条件下的真实反应。实验表明，我们的模型优于现有反应生成方法。代码与数据将公开于 https://ereact.github.io/
原文摘要 · Abstract (English)
Emotion serves as an essential component in daily human interactions. Existing human motion generation frameworks do not consider the impact of emotions, which reduces naturalness and limits their application in interactive tasks, such as human reaction synthesis. In this work, we introduce a novel task: generating diverse reaction motions in response to different emotional cues. However, learning emotion representation from limited motion data and incorporating it into a motion generation framework remains a challenging problem. To address the above obstacles, we introduce a semi-supervised emotion prior in an actor-reactor diffusion model to facilitate emotion-driven reaction synthesis. Specifically, based on the observation that motion clips within a short sequence tend to share the same emotion, we first devise a semi-supervised learning framework to train an emotion prior. With this prior, we further train an actor-reactor diffusion model to generate reactions by considering both spatial interaction and emotional response. Finally, given a motion sequence of an actor, our approach can generate realistic reactions under various emotional conditions. Experimental results demonstrate that our model outperforms existing reaction generation methods. The code and data will be made publicly available at https://ereact.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。