arXiv:2505.15065cs.CLcs.AI2025-05EMNLP被引 9

小模型也能共情:评估0.5B-5B参数模型在创伤后应激障碍对话中的共情能力

The Pursuit of Empathy: Evaluating Small Language Models for PTSD Dialogue Support

  • 构建包含500个临床背景角色的TIDE数据集,用于评估小模型共情表现
  • 微调后模型共情度提升,部分小模型接近人类评分水平
  • 适用于资源有限但需情感智能的心理健康支持系统

本文研究了参数量在0.5B至5B之间的小语言模型在为创伤后应激障碍(PTSD)患者提供对话支持时的共情能力。我们引入了创伤知情共情对话(TIDE)数据集,包含10,000条两轮对话,涵盖500个多样化的、基于临床的PTSD人物设定(https://huggingface.co/datasets/yenopoya/TIDE)。以前沿模型输出作为真实标签,在零样本和微调条件下评估了八个小规模LLM。微调显著提升了共情能力,改善了余弦相似度与感知共情度,但不同情绪场景下增益差异明显,且小模型存在“知识迁移上限”。如预期,Claude Sonnet 3.5始终领先,但令人惊讶的是,部分小模型表现接近人类共情评分。人口统计学分析显示,老年用户更偏好先确认痛苦再提供支持的回应(p = .004),研究生教育程度用户在特定情境中更倾向情感丰富的回复。性别差异不显著(p > 0.15),表明设计普适性共情模型具有可行性。本研究为构建高效、情感智能的心理健康支持系统提供了关键洞见。

原文摘要 · Abstract (English)

This paper investigates the capacity of small language models (0.5B-5B parameters) to generate empathetic responses for individuals with PTSD. We introduce Trauma-Informed Dialogue for Empathy (TIDE), a novel dataset comprising 10,000 two-turn conversations across 500 diverse, clinically-grounded PTSD personas (https://huggingface.co/datasets/yenopoya/TIDE). Using frontier model outputs as ground truth, we evaluate eight small LLMs in zero-shot settings and after fine-tuning. Fine-tuning enhances empathetic capabilities, improving cosine similarity and perceived empathy, although gains vary across emotional scenarios and smaller models exhibit a "knowledge transfer ceiling." As expected, Claude Sonnet 3.5 consistently outperforms all models, but surprisingly, the smaller models often approach human-rated empathy levels. Demographic analyses showed that older adults favored responses that validated distress before offering support (p = .004), while graduate-educated users preferred emotionally layered replies in specific scenarios. Gender-based differences were minimal (p > 0.15), suggesting the feasibility of broadly empathetic model designs. This work offers insights into building resource-efficient, emotionally intelligent systems for mental health support.

共情对话小模型心理健康PTSD

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。