让AI克隆会听：加入回应和点头，更像真人。
Does Listening Matter? Backchanneling and Nodding in AI Clone
- 用实时模型驱动语音回应和点头动作
- 35人测试显示感知专注度显著提升
- 适合做虚拟助手或数字人交互的开发者
AI克隆通常只模仿说话内容和声音，却忽略倾听行为。本文研究在克隆中加入实时预测的语音回应和头部点头是否能增强其存在感与真实感。在35名参与者参与的组内实验中，加入这些互动行为后，用户感知到的注意力水平、与真实人物对话的感觉以及共现感均显著提升。结果表明，提升AI克隆的真实感需超越语音和回复内容，扩展至交互式倾听行为。
原文摘要 · Abstract (English)
AI clones that imitate a specific person typically reproduce what the person says and how they sound, but not how they listen. We investigate whether adding multimodal listening behaviors gives such a clone more presence and authenticity. We integrated verbal backchannels and head nodding, driven by real-time prediction models, into an AI clone equipped with voice cloning and LLM-based responses. In a within-subjects study (N=35), adding these behaviors significantly improved the perceived attentiveness of the avatar, the sense of talking with the real person, and the feeling of co-presence. These results indicate that AI clone fidelity should extend beyond voice and response content to include interactive listening behavior.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。