用中文、印地语等生成心理对话数据,发现单纯换语言难保证临床一致性。
Creating Multilingual Mental Health Dialogue Datasets: Limits of Persona-Based Localization via Nationality and Language

- 通过调整人物国籍与语言参数生成多语言心理对话数据
- 非英语文本中大模型评估抑郁程度准确率下降,表现差异大
- 提醒需文化适配数据生成,避免英语中心偏见
人工智能与大型语言模型(LLMs)在应对全球心理健康挑战方面展现出巨大潜力。尽管问题具有全球性,高质量训练与评估数据仍严重不足。为弥补这一缺口,研究者常生成合成临床人物角色以模拟用户数据并测试数字心理健康支持系统。然而,现有验证过的人物角色大多基于英语语境。本文探究是否可将类似方法用于生成多语言心理健康数据集。我们修改人物角色的国籍与语言参数,生成了中文、孟加拉语和印地语的临床对话数据,并评估不同LLM在判断这些多语言数据中抑郁严重程度的表现,与英语基线对比。结果表明,仅通过增加国籍和语言参数可能不足以确保跨语言的临床一致性;多个LLM在评估非英语文本时出现偏差,性能因模型而异。这揭示了将英语中心人物角色应用于多语言情境的系统性局限。研究强调,亟需文化敏感的数据生成方式,以实现全球范围内心理健康系统的公平性。
原文摘要 · Abstract (English)
AI and large language models (LLMs) have emerged as promising tools to address global mental health challenges. Despite the global nature of these challenges, there remains a critical shortage of high-quality datasets for training and evaluating such systems. To mitigate this gap, researchers increasingly generate synthetic clinical personas to simulate user data and test digital mental health support systems. However, most validated personas rely on English-centric contexts. This paper investigates whether similar persona-based methods can be used to generate multilingual mental health datasets. We modified nationality and language parameters in personas to generate clinical dialogues in Mandarin, Bengali, and Hindi. We then examined how different LLMs perform when evaluating the depression severity of these generated multilingual datasets against the baseline in English. Our findings indicate that just adding nationality and language parameters in personas might not be adequate, as it can introduce clinical inconsistency across languages. LLM judge models often exhibit inaccuracies in assessing depression severity in non-English texts, with performance varying across different models. This exposes the systemic limitations of applying English-centric personas to multilingual contexts. Ultimately, our work highlights the urgent need for culturally responsive data generation to ensure equitable mental health systems globally.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。