让机器人通过表情和语言实时感知情绪,实现更贴心的对话。
PERCY: Personal Emotional Robotic Conversational System
- 结合人脸表情与话语情感,动态调整回应策略。
- 在多轮对话中表现高连贯性、相关性和多样性。
- 适合需要长期互动的社交机器人场景。
传统规则驱动的对话机器人受限于预设脚本和静态响应映射,难以适应个性化、长期的人机交互。尽管GPT-4等大语言模型在开放域对话上取得突破,但现有社交机器人仍缺乏情感感知与持续个性化能力,限制了多轮交互中的参与度。本文提出PERCY(Personal Emotional Robotic Conversational sYstem),一个基于ROS的多模态框架,通过细调GPT-4推理引擎,融合文本情感分析与视觉情绪线索,实时评估并响应用户情绪状态,支持开放域、多轮对话。在多种对话质量指标下,PERCY展现出强连贯性、相关性和多样性。人类评估表明其个性化能力优于现有模型,自然度可与主流模型媲美。该工作展示了将先进多模态感知与个性化技术融入社交机器人对话系统的潜力。
原文摘要 · Abstract (English)
Traditional rule-based conversational robots, constrained by predefined scripts and static response mappings, fundamentally lack adaptability for personalized, long-term human interaction. While Large Language Models (LLMs) like GPT-4 have revolutionized conversational AI through open-domain capabilities, current social robots implementing LLMs still lack emotional awareness and continuous personalization. This dual limitation hinders their ability to sustain engagement across multiple interaction sessions. We bridge this gap with PERCY (Personal Emotional Robotic Conversational sYstem), a system designed to enable open-domain, multi-turn dialogues by dynamically analyzing users' real-time facial expressions and vocabulary to tailor responses based on their emotional state. Built on a ROS-based multimodal framework, PERCY integrates a fine-tuned GPT-4 reasoning engine, combining textual sentiment analysis with visual emotional cues to accurately assess and respond to user emotions. We evaluated PERCY's performance through various dialogue quality metrics, showing strong coherence, relevance, and diversity. Human evaluations revealed PERCY's superior personalization and comparable naturalness to other models. This work highlights the potential for integrating advanced multimodal perception and personalization in social robot dialogue systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。