用检索增强生成提升心理健康AI的意图识别准确率
Retrieval-Augmented Generation in LLMs for Mental Health: Quantifying the Incremental Contribution of Retrieval Within a Layered Safety Architecture
- 在安全架构中引入检索增强生成，补充LLM上下文
- RAG使意图识别准确率提升，召回率显著提高
- 适合关注AI心理干预安全与可靠性的研究者
数字心理健康干预（DMHIs）可规模化提供支持，但在情绪波动情境下准确识别用户意图仍具挑战。纯参数化大语言模型（LLMs）缺乏专门的安全架构，可能遗漏关键线索或产生幻觉，影响可靠性。检索增强生成（RAG）通过引入外部检索上下文，可提升危机情境下的意图识别能力。现有商业化DMHIs通常采用规则过滤、符号升级协议和神经分类等多层独立安全机制，但各层增量贡献未被量化。本文在Wysa系统中对比了六种LLM在开启与关闭RAG模式下的表现，使用匿名真实与合成对话数据，由专业临床团队标注多类意图（如自伤、虐待、恐慌）。评估指标包括准确率、召回率、精确率和F1值，并进行统计显著性检验。结果表明，尽管RAG导致误报增加，但符合安全优先原则——高敏感度确保风险案例被送审而非直接执行。整体支持RAG作为提升LLM驱动DMHIs准确性、一致性和安全性的有效方法。
原文摘要 · Abstract (English)
Digital mental health interventions (DMHIs) offer scalable support, but ensuring they accurately detect users' intent during volatile situations can be challenging. Pure parametric Large Language models (LLMs) do not contain specific safety critical architecture, and can miss critical cues, or hallucinate, undermining reliability. Retrieval Augmented Generation (RAG), which supplements an LLM with retrieved context, could enhance intent detection during volatile situations. Commercially available DMHIs typically combine multiple independent safety layers like rule-based filters, symbolic escalation protocols, and neural classification. The incremental contribution of any single layer, however, remains unquantified. This paper evaluates six LLM models within a DMHI called Wysa, via a controlled comparison of RAG-enabled versus RAG-disabled modes. Anonymized real and synthetic user-chatbot exchanges were annotated by a qualified clinical team against multi-class intent categories (e.g. self-harm, abuse, panic). The study computed classification accuracy, recall, precision and F1 scores against ground truth labels and tested differences for statistical significance. Performance was also examined by risk category and inter-model agreement. While RAG caused a rise in false alarms, the trade-off is consistent with safety-critical design principles that prioritize sensitivity, where flagged cases are routed to additional review rather than acted on directly. Overall, these findings support RAG as a promising approach to improve the accuracy, consistency and safety of LLM-driven DMHIs. Keywords: Digital Mental Health Intervention, Large Language Model, Retrieval Augmented Generation, Accuracy, Recall, Precision
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。