基于真实抑郁经历，揭示AI心理聊天机器人设计的关键价值与风险
AI Chatbots for Mental Health: Values and Harms from Lived Experiences of Depression
- 通过自研聊天机器人Zenny，模拟抑郁自我管理场景
- 17位亲历者指出信息支持、情感陪伴等五项核心价值
- 为减少伤害提供可落地的设计建议，适合心理健康AI开发者
大型语言模型的进展使聊天机器人能够应对各类敏感议题，包括心理健康问题。尽管其有效性与可靠性仍存疑，相关开发仍在加速，可能带来潜在危害。为更好识别并缓解这些风险，理解具有真实抑郁经历人群的价值观至关重要。本研究设计了一款基于GPT-4o的科技探针Zenny，让参与者在基于前期研究的抑郁自我管理情境中互动。通过对17位有抑郁经历者进行访谈，主题分析提炼出五大核心价值：信息支持、情感支持、个性化、隐私保护和危机应对。本研究探讨了这些价值观与潜在危害之间的关系，提出面向心理健康AI聊天机器人的设计建议，旨在提升自我管理支持能力的同时降低风险。
原文摘要 · Abstract (English)
Recent advancements in LLMs enable chatbots to interact with individuals on a range of queries, including sensitive mental health contexts. Despite uncertainties about their effectiveness and reliability, the development of LLMs in these areas is growing, potentially leading to harms. To better identify and mitigate these harms, it is critical to understand how the values of people with lived experiences relate to the harms. In this study, we developed a technology probe, a GPT-4o based chatbot called Zenny, enabling participants to engage with depression self-management scenarios informed by previous research. We used Zenny to interview 17 individuals with lived experiences of depression. Our thematic analysis revealed key values: informational support, emotional support, personalization, privacy, and crisis management. This work explores the relationship between lived experience values, potential harms, and design recommendations for mental health AI chatbots, aiming to enhance self-management support while minimizing risks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。