让AI像真人演员一样,根据角色和情境即兴说出台词。
ActorMind: Emulating Human Actor Reasoning for Speech Role-Playing

- 用多智能体框架模拟演员从理解角色到表达情绪的全过程。
- 在包含7653句对话、313个场景的ActorMindBench上验证有效。
- 适合做语音交互、虚拟人、戏剧化对话系统的研究者。
角色扮演因能促进人机交互与社会学研究而日益受到关注,但现有工作局限于文本模态,忽视了日常交流中占主导地位的语音,限制了真实角色扮演的发展。为弥合这一差距,我们提出面向语音角色扮演的基准测试平台ActorMindBench,并设计相应的推理框架ActorMind。具体而言,语音角色扮演使模型能基于角色、场景与对话内容,生成带有个性化语音特征的自然响应。ActorMindBench是一个分层基准,包含7,653条语句(语句级)、313个场景(场景级)和6个角色(角色级)。ActorMind是一种即插即用的多智能体链式思维推理框架,模拟人类演员在剧场中的表现过程:首先由Eye Agent阅读角色描述,再由Ear Agent理解上下文对话中的情感线索,随后Brain Agent生成情感状态描述,最后Mouth Agent输出带有相应情绪特征的台词。实验表明,ActorMind能显著提升语音角色扮演的表现效果。
原文摘要 · Abstract (English)
Role-playing has garnered rising attention as it provides a strong foundation for human-machine interaction and facilitates sociological research. However, current work is confined to textual modalities, neglecting speech, which plays a predominant role in daily life, thus limiting genuine role-playing. To bridge this gap, we conceptualize and benchmark speech role-playing through ActorMindBench, and we present a corresponding reasoning framework, called ActorMind. Specifically, (1) Speech Role-Playing enables models to deliver spontaneous responses with personalized verbal traits based on their role, the scene, and spoken dialogue. (2) ActorMindBench is a hierarchical benchmark comprises Utterance-Level content with 7,653 utterances, Scene-Level content with 313 scenes, and Role-Level content with 6 roles. (3) ActorMind is an off-the-shelf, multi-agent, chain-of-though style reasoning framework that emulates how human actors perform in theaters. Concretely, ActorMind first reads its assigned role description via Eye Agent, then comprehends emotional cues within contextual spoken dialogues through Ear Agent. Subsequently, Brain Agent generates a descriptive emotional state, and finally, Mouth Agent delivers the scripts infused with corresponding emotion state. Experimental results demonstrate the effectiveness of ActorMind in enhancing speech role-playing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。