用多个专家角色协作生成逼真心理咨询对话,提升数据质量与评估一致性。
MAGneT: Coordinated Multi-Agent Generation of Synthetic Multi-Turn Mental Health Counseling Sessions
- 拆分咨询任务由不同LLM角色协同完成,模拟真实心理咨询流程。
- 专家评分显示,生成对话在9个维度上优于基线方法77.2%。
- 生成数据可有效微调模型,使认知疗法表现提升6.9%以上。
心理咨询服务需求增长迅速,但高质量、隐私合规的训练数据仍稀缺。本文提出MAGneT,一种多智能体框架,通过专业化分工的LLM代理协作生成合成多轮心理咨询对话,每类代理专注一种核心心理技术。相比传统单代理方法,MAGneT更准确捕捉真实咨询的结构与细节。我们还设计统一评估框架,整合多种自动指标,并将专家评估维度从4项扩展至9项,解决以往评估不一致问题。实验表明,MAGneT显著优于现有方法:专家在9项维度中平均77.2%偏好MAGneT生成内容;使用Llama3-8B-Instruct作为主干模型时,生成对话在认知治疗评分量表(CTRS)上实现3.2%的通用咨询技能提升和4.3%的CBT专项技能提升。基于MAGneT生成数据微调的开源模型,在CTRS上比基线合成数据微调模型平均高出6.9%。代码、数据及微调模型已开源。
原文摘要 · Abstract (English)
The growing demand for scalable psychological counseling highlights the need for high-quality, privacy-compliant data, yet such data remains scarce. Here we introduce MAGneT, a novel multi-agent framework for synthetic psychological counseling session generation that decomposes counselor response generation into coordinated sub-tasks handled by specialized LLM agents, each modeling a key psychological technique. Unlike prior single-agent approaches, MAGneT better captures the structure and nuance of real counseling. We further propose a unified evaluation framework that consolidates diverse automatic metrics and expands expert assessment from four to nine counseling aspects, thus addressing inconsistencies in prior evaluation protocols. Empirically, MAGneT substantially outperforms existing methods: experts prefer MAGneT-generated sessions in 77.2% of cases on average across the nine aspects over the strongest baseline, and sessions generated by MAGneT using Llama3-8B-Instruct backbone yield 3.2% higher general counseling skills and 4.3% higher CBT-specific skills on cognitive therapy rating scale (CTRS). An open source Llama3-8B-Instruct model fine-tuned on MAGneT-generated data also outperforms models fine-tuned using baseline synthetic datasets by 6.9% on average on CTRS. We make our code, data and fine-tuned model public.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。