用MetaHuman生成逼真非语言行为，支持真实场景互动研究
Human-like Nonverbal Behavior with MetaHumans in Real-World Interaction Studies: An Architecture Using Generative Methods and Motion Capture
- 基于MetaHuman与动作捕捉的分布式架构，融合生成与手工动画
- 在德国博物馆开展三周实地研究，验证真实交互中非语言行为效果
- 适合人机交互、虚拟助手、社交机器人等领域的研究人员
社交互动代理在医疗、教育和服务等领域日益重要，尤其虚拟代理因可扩展性强备受关注。为实现自然交互，系统需具备面部表情、手势等非语言行为能力。尽管自然语言处理技术快速发展，但在真实场景中融入类人非语言行为仍属空白。本文提出一种分布式架构，结合Epic Games MetaHuman、先进对话AI与摄像头用户管理，支持动作捕捉、手工动画及生成式方法构建非语言行为。该系统在德国波恩博物馆开展为期三周的实地研究，展示了其在真实非语言行为研究中的潜力。
原文摘要 · Abstract (English)
Socially interactive agents are gaining prominence in domains like healthcare, education, and service contexts, particularly virtual agents due to their inherent scalability. To facilitate authentic interactions, these systems require verbal and nonverbal communication through e.g., facial expressions and gestures. While natural language processing technologies have rapidly advanced, incorporating human-like nonverbal behavior into real-world interaction contexts is crucial for enhancing the success of communication, yet this area remains underexplored. One barrier is creating autonomous systems with sophisticated conversational abilities that integrate human-like nonverbal behavior. This paper presents a distributed architecture using Epic Games MetaHuman, combined with advanced conversational AI and camera-based user management, that supports methods like motion capture, handcrafted animation, and generative approaches for nonverbal behavior. We share insights into a system architecture designed to investigate nonverbal behavior in socially interactive agents, deployed in a three-week field study in the Deutsches Museum Bonn, showcasing its potential in realistic nonverbal behavior research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。