arXiv:2505.20521cs.AIcs.CL2025-05被引 5

让五个情绪角色协作写回复,让AI更懂情感和沟通

Project Riley: Multimodal Multi-Agent LLM Collaboration with Emotional Reasoning and Voting

论文配图:Project Riley: Multimodal Multi-Agent LLM Collaboration with Emotional Reasoning and Voting
图 1 · 摘自论文原文
  • 用五种情绪角色分轮对话,共同生成与修正回答
  • 用户测试显示情绪匹配度高,表达清晰且自然
  • 适合需要共情能力的紧急场景或情感化交互

本文提出Project Riley,一种面向情感驱动推理的多模态多智能体对话架构。受皮克斯《头脑特工队》启发,系统包含快乐、悲伤、恐惧、愤怒、厌恶五种情绪代理,通过结构化多轮对话生成、批判并迭代优化回复。最终推理机制整合各代理贡献,输出反映主导情绪或融合多视角的连贯结果。系统结合文本与视觉大模型,融入先进推理与自我优化流程。首个原型在本地离线环境部署,注重情感表现力与计算效率。由此衍生出名为Armando的改进版,用于应急场景,通过检索增强生成(RAG)与累积上下文追踪,提供情感适配且事实准确的信息。项目原型经用户测试,参与者完成评估情绪恰当性、清晰度与实用性、自然度与拟人化的问卷。结果显示在结构化场景中表现优异,尤其在情绪一致性与沟通清晰度方面突出。

原文摘要 · Abstract (English)

This paper presents Project Riley, a novel multimodal and multi-model conversational AI architecture oriented towards the simulation of reasoning influenced by emotional states. Drawing inspiration from Pixar's Inside Out, the system comprises five distinct emotional agents - Joy, Sadness, Fear, Anger, and Disgust - that engage in structured multi-round dialogues to generate, criticise, and iteratively refine responses. A final reasoning mechanism synthesises the contributions of these agents into a coherent output that either reflects the dominant emotion or integrates multiple perspectives. The architecture incorporates both textual and visual large language models (LLMs), alongside advanced reasoning and self-refinement processes. A functional prototype was deployed locally in an offline environment, optimised for emotional expressiveness and computational efficiency. From this initial prototype, another one emerged, called Armando, which was developed for use in emergency contexts, delivering emotionally calibrated and factually accurate information through the integration of Retrieval-Augmented Generation (RAG) and cumulative context tracking. The Project Riley prototype was evaluated through user testing, in which participants interacted with the chatbot and completed a structured questionnaire assessing three dimensions: Emotional Appropriateness, Clarity and Utility, and Naturalness and Human-likeness. The results indicate strong performance in structured scenarios, particularly with respect to emotional alignment and communicative clarity.

多智能体情感推理对话系统RAG

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。