用户误信通用大模型聊天工具能像医生一样保密，实则存在隐私漏洞。
Exploring User Security and Privacy Attitudes and Concerns Toward the Use of General-Purpose LLM Chatbots for Mental Health
- 通过访谈发现用户混淆了AI情感回应与真实责任归属。
- 超半数用户错误认为聊天记录受医疗隐私法规保护。
- 提出'无形脆弱性'概念，强调心理信息易被忽视的风险。
越来越多用户依赖大型语言模型（LLM）驱动的对话代理获取情感支持。尽管已有研究关注专为心理健康设计的聊天机器人的隐私与安全问题，但这些系统大多为规则驱动型，未利用生成式AI。目前尚缺乏针对用户使用通用型LLM聊天工具进行心理管理时的隐私与安全关切、态度及期望的实证研究。通过对21名美国参与者开展半结构化访谈，我们发现存在关键误解与普遍风险意识缺失：用户将LLM表现出的人类式共情等同于人类责任，并错误认为其与聊天机器人交流受到与持证治疗师相同的法规（如HIPAA）保护。我们提出'无形脆弱性'概念，即情绪或心理披露相较于财务或位置等有形数据更易被低估。为此，我们提出了更有效地保护用户心理披露的建议。
原文摘要 · Abstract (English)
Individuals are increasingly relying on large language model (LLM)-enabled conversational agents for emotional support. While prior research has examined privacy and security issues in chatbots specifically designed for mental health purposes, these chatbots are overwhelmingly "rule-based" offerings that do not leverage generative AI. Little empirical research currently measures users' privacy and security concerns, attitudes, and expectations when using general-purpose LLM-enabled chatbots to manage and improve mental health. Through 21 semi-structured interviews with U.S. participants, we identified critical misconceptions and a general lack of risk awareness. Participants conflated the human-like empathy exhibited by LLMs with human-like accountability and mistakenly believed that their interactions with these chatbots were safeguarded by the same regulations (e.g., HIPAA) as disclosures with a licensed therapist. We introduce the concept of "intangible vulnerability," where emotional or psychological disclosures are undervalued compared to more tangible forms of information (e.g., financial or location-based data). To address this, we propose recommendations to safeguard user mental health disclosures with general-purpose LLM-enabled chatbots more effectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。