让机器人通过推理和多模态输入,从第一次对话就开始个性化交互。
Reasoning LLMs for User-Aware Multimodal Conversational Agents
- 用思维链推理模型动态推断用户偏好,结合多模态输入初始化用户画像。
- 在ElderlyTech-VQA测试中,ROUGE-1提升23.2%,对老年人互动效果更佳。
- 适合需要快速建立用户理解的智能服务场景,尤其关注老年群体体验。
社交机器人中的个性化对提升人机交互效果至关重要,但系统常面临冷启动问题——初始阶段缺乏用户偏好或特征信息。本文提出USER-LLM R1框架,通过动态用户建模与模型初始化解决此挑战。该方法融合思维链(CoT)推理模型,迭代推断用户偏好;结合视觉语言模型(VLMs),从多模态输入中初始化用户画像,实现首次交互即个性化。借助检索增强生成(RAG)架构,系统在内在的思维链过程中动态优化用户表征,确保响应具备上下文相关性与适应性。在ElderlyTech-VQA基准上的评估显示,相比最先进基线,ROUGE-1提升23.2%,ROUGE-2提升0.6%,ROUGE-L提升8%。消融实验表明推理模型规模对性能有显著影响。人工评估进一步验证了该框架的有效性,尤其在老年用户中提升了参与度与信任感。伦理方面,重点讨论并解决了隐私保护与偏见缓解问题,保障负责任部署。
原文摘要 · Abstract (English)
Personalization in social robotics is critical for fostering effective human-robot interactions, yet systems often face the cold start problem, where initial user preferences or characteristics are unavailable. This paper proposes a novel framework called USER-LLM R1 for a user-aware conversational agent that addresses this challenge through dynamic user profiling and model initiation. Our approach integrates chain-of-thought (CoT) reasoning models to iteratively infer user preferences and vision-language models (VLMs) to initialize user profiles from multimodal inputs, enabling personalized interactions from the first encounter. Leveraging a Retrieval-Augmented Generation (RAG) architecture, the system dynamically refines user representations within an inherent CoT process, ensuring contextually relevant and adaptive responses. Evaluations on the ElderlyTech-VQA Bench demonstrate significant improvements in ROUGE-1 (+23.2%), ROUGE-2 (+0.6%), and ROUGE-L (+8%) F1 scores over state-of-the-art baselines, with ablation studies underscoring the impact of reasoning model size on performance. Human evaluations further validate the framework's efficacy, particularly for elderly users, where tailored responses enhance engagement and trust. Ethical considerations, including privacy preservation and bias mitigation, are rigorously discussed and addressed to ensure responsible deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。