用AI精准评估心理咨询中应对来访者抗拒的回应质量并给出解释。
Multi-dimensional Assessment and Explainable Feedback for Counselor Responses to Client Resistance in Text-based Counseling with LLMs
- 拆解咨询回应为四种沟通机制,构建可量化的评估框架。
- 模型在识别不同机制质量上达到77-81%准确率,显著优于GPT-4o等大模型。
- 生成的反馈解释贴近专家思路,实测能有效提升咨询师应对能力。
在文本心理辅导中,有效应对来访者的抗拒是高阶临床技能,但从业者常缺乏及时且可扩展的督导反馈。现有NLP研究虽关注整体咨询质量与通用治疗技巧,却未能细致评估来访者表现出抗拒的关键时刻。本文提出一个面向文本咨询中应对客户抗拒行为的多维度评估流程,引入基于理论的框架,将咨询师回应分解为四类沟通机制。基于该框架,我们构建并公开了一个由专家标注的真实咨询语料数据集,包含咨询师-来访者对话、专业评分及解释性理由。利用该数据,我们在Llama-3.1-8B-Instruct基础上进行全参数指令微调,以建模细粒度回应质量判断并生成解释。实验表明,该方法在区分不同沟通机制质量方面取得77-81%的F1分数,远超GPT-4o和Claude-3.5-Sonnet(45-59% F1)。模型生成的解释与专家参考高度一致,人类专家评分达2.8-2.9/3.0,接近满分。对43名咨询师的对照实验进一步验证,接收该AI反馈后,其应对客户抗拒的能力显著提升。
原文摘要 · Abstract (English)
Effectively addressing client resistance is a sophisticated clinical skill in psychological counseling, yet practitioners often lack timely and scalable supervisory feedback to refine their approaches. Although current NLP research has examined overall counseling quality and general therapeutic skills, it fails to provide granular evaluations of high-stakes moments where clients exhibit resistance. In this work, we present a comprehensive pipeline for the multi-dimensional evaluation of human counselors' interventions specifically targeting client resistance in text-based therapy. We introduce a theory-driven framework that decomposes counselor responses into four distinct communication mechanisms. Leveraging this framework, we curate and share an expert-annotated dataset of real-world counseling excerpts, pairing counselor-client interactions with professional ratings and explanatory rationales. Using this data, we perform full-parameter instruction tuning on a Llama-3.1-8B-Instruct backbone to model fine-grained evaluative judgments of response quality and generate explanations underlying. Experimental results show that our approach can effectively distinguish the quality of different communication mechanisms (77-81% F1), substantially outperforming GPT-4o and Claude-3.5-Sonnet (45-59% F1). Moreover, the model produces high-quality explanations that closely align with expert references and receive near-ceiling ratings from human experts (2.8-2.9/3.0). A controlled experiment with 43 counselors further confirms that receiving these AI-generated feedback significantly improves counselors' ability to respond effectively to client resistance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。