让医疗AI根据用户情绪调整回应，既贴心又准确。
EmoMed: An Emotionally-Aware Agent for Multimodal Medical Support with Real-Time Information Retrieval
- 通过分析文字和图像识别焦虑等情绪，动态调整回复语气和内容。
- 在7个主流大模型上测试，情感自适应响应显著提升用户体验。
- 适合需要高共情力的远程医疗、心理健康支持场景。
我们提出EmoMed——一种多模态医疗咨询智能体，能根据用户情绪状态动态调整回应，同时保证临床准确性。该系统处理文本与医学图像，从用户输入中检测焦虑、困惑、紧急等情感指标，并相应调节回复的语气、结构和详尽程度。为确保信息可靠性，采用双检索机制：基于网络的实时事实核查与持续更新的API连接医学知识库。我们在七个顶尖语言模型（GPT-4/5、Qwen3、Llama 4、Gemini 2.5、Grok4、Claude3）上进行评估，使用多项指标包括LLM-as-judge评分、MedQA式准确率测试、BERT Score、安全性和帮助性评分以及多模态医疗基准。结果表明，情感自适应回应在各项评估中均优于中性基线，且不牺牲临床准确性。受控用户研究验证了上述发现，参与者普遍报告感知到更高同理心与沟通清晰度，同时维持对事实准确性的信任。源代码见：https://github.com/NasonovIvan/EmoMed-Agent
原文摘要 · Abstract (English)
We present EmoMed - a multimodal medical consultation agent that adapts its responses based on users' emotional states while maintaining clinical accuracy. The system processes text and medical images, detects affect indicators (anxiety, confusion, urgency) from user input, and adjusts response tone, structure, and detail level accordingly. To ensure factual reliability, the agent grounds clinical information through a dual retrieval mechanism: web-based fact-checking and an API-connected, continuously updated medical knowledge base. We evaluate our approach across seven state-of-the-art language models (GPT-4/5, Qwen3, Llama 4, Gemini 2.5, Grok4, Claude3) using comprehensive metrics including LLM-as-judge assessments, MedQA style accuracy tests, BERT Score, safety/helpfulness ratings, and multimodal medical benchmarks. The results demonstrate that emotionally adaptive responses consistently outperform neutral baseline across evaluation dimensions, without compromising clinical accuracy. A controlled user study validated these findings, with participants reporting improved perceived empathy and communication clarity, while maintaining trust in factual accuracy. Source code: https://github.com/NasonovIvan/EmoMed-Agent
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。