用分阶段模拟客户评估戒烟咨询中的动机访谈质量
Evaluation of Motivational Interviewing Counsellors with Task-Aware Multi-Stage LLM-Based Simulated Clients
- 设计三阶段对话流程，聚焦激发矛盾心理的核心任务
- 相比旧模型，能更好区分咨询水平，减少信息过早暴露
- 适合评估大模型心理咨询助手，推动疗法标准化
大型语言模型（LLM）在动机访谈（MI）咨询师的开发与评测中广泛应用，但现有基于LLM的模拟客户未能贴合MI疗法的核心任务。关键任务之一是‘激发’（evoking），即先引出客户的矛盾心理，再增强其改变动机。本文提出Evoke-Sim，一种面向吸烟戒断场景、任务感知的多阶段LLM模拟客户框架，专为激发任务设计。该框架采用结构化客户档案、针对激发任务的三阶段对话流程，以及控制各阶段披露信息的揭示策略。实验表明，相较于现有基于档案的模拟客户，Evoke-Sim在使用任务感知评价指标时，更能有效区分不同质量的MI咨询表现，同时显著减少非依据档案的陈述和过早披露客户信息，为评估基于LLM的MI咨询师设立了更高标准。
原文摘要 · Abstract (English)
The development and benchmarking of Large Language Model (LLM)-based Motivational Interviewing (MI) counsellors now often rely on LLM-based simulated clients. Prior work on simulated clients, however, has not aligned with the specific tasks fundamental to the MI therapy approach. A key task is evoking, in which the counsellor first elicits the client's ambivalence and then strengthens the client's motivation for change. We present Evoke-Sim, a task-aware, multi-stage LLM-based client simulation framework for evaluating MI counsellors in smoking cessation, designed specifically for the evoking MI task. Evoke-Sim employs structured client profiles, an evoking-specific three-stage conversation flow, and a reveal policy that regulates which client profile information might be disclosed at each stage. We show that compared to existing profile-grounded simulated clients, Evoke-Sim is better at differentiating levels of MI quality using task-aware evaluation metrics, while reducing non-grounded client statements and premature disclosure of client information, setting a higher standard for the evaluation of LLM-based MI counsellors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。