测试大模型在心理治疗中的伦理判断能力，发现其易被滥用但可改进。
When LLM Therapists Become Salespeople: Evaluating Large Language Models for Ethical Motivational Interviewing
- 用链式伦理提示提升模型对治疗伦理的理解
- 模型对非道德对话识别率不足50%，易生成不当建议
- 适合关注AI心理健康应用安全的研究者与开发者
大型语言模型（LLMs）在心理健康领域应用广泛，尤其在动机访谈（MI）方面展现潜力。然而，现有研究缺乏对其理解MI伦理能力的评估。鉴于恶意使用者可能利用模型实施非道德的动机访谈，评估其区分伦理与非伦理行为的能力至关重要。本研究通过多项实验考察了LLMs在MI中的伦理意识。结果显示，模型对MI知识掌握程度中等至较高，但其伦理标准与MI核心精神不一致，既会生成非道德回应，也难以识别他人提出的非道德内容。为此，我们提出一种链式伦理提示（Chain-of-Ethic prompt）以缓解风险并提升安全性。实验表明，该策略显著提升了模型生成和检测伦理响应的能力。研究强调，必须建立针对伦理型LLM心理治疗系统的安全评估与规范指导。
原文摘要 · Abstract (English)
Large language models (LLMs) have been actively applied in the mental health field. Recent research shows the promise of LLMs in applying psychotherapy, especially motivational interviewing (MI). However, there is a lack of studies investigating how language models understand MI ethics. Given the risks that malicious actors can use language models to apply MI for unethical purposes, it is important to evaluate their capability of differentiating ethical and unethical MI practices. Thus, this study investigates the ethical awareness of LLMs in MI with multiple experiments. Our findings show that LLMs have a moderate to strong level of knowledge in MI. However, their ethical standards are not aligned with the MI spirit, as they generated unethical responses and performed poorly in detecting unethical responses. We proposed a Chain-of-Ethic prompt to mitigate those risks and improve safety. Finally, our proposed strategy effectively improved ethical MI response generation and detection performance. These findings highlight the need for safety evaluations and guidelines for building ethical LLM-powered psychotherapy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。