用聊天机器人缓解孤独感，可能适得其反。
If Eleanor Rigby Had Met ChatGPT: A Study on Loneliness in a Post-LLM World
- 分析用户在非任务场景下与ChatGPT的对话，发现37%涉及情感求助。
- 面对自杀念头等敏感话题，模型响应失效，有毒内容发生率高出35%。
- 女性遭针对性攻击概率是男性的22倍，提示潜在伦理风险。
警告：本文涉及暴力、性及自杀相关内容。孤独感，即缺乏有意义关系的状态，严重影响身心健康，且全球普遍。以往研究认为大语言模型（LLMs）可能缓解孤独感。但本文指出，像ChatGPT这类广泛使用的LLM服务虽被用于此目的，却非为此设计，存在更高风险。我们分析了用户在非任务导向场景下与ChatGPT的交互。在被归类为孤独的对话中，37%的用户寻求建议或情感确认，模型提供了良好互动。然而，在应对自杀倾向或创伤等敏感情境时，模型表现失败。同时观察到有毒内容发生率上升35%，其中女性被针对的可能性是男性的22倍。研究凸显该技术带来的伦理与法律挑战，如激进化或加剧孤立。最后提出对学术界与产业界的改进建议。
原文摘要 · Abstract (English)
Warning: this paper discusses content related, but not limited to, violence, sex, and suicide. Loneliness, or the lack of fulfilling relationships, significantly impacts a person's mental and physical well-being and is prevalent worldwide. Previous research suggests that large language models (LLMs) may help mitigate loneliness. However, we argue that the use of widespread LLMs in services like ChatGPT is more prevalent--and riskier, as they are not designed for this purpose. To explore this, we analysed user interactions with ChatGPT outside of its marketed use as a task-oriented assistant. In dialogues classified as lonely, users frequently (37%) sought advice or validation, and received good engagement. However, ChatGPT failed in sensitive scenarios, like responding appropriately to suicidal ideation or trauma. We also observed a 35% higher incidence of toxic content, with women being 22x more likely to be targeted than men. Our findings underscore ethical and legal questions about this technology, and note risks like radicalisation or further isolation. We conclude with recommendations to research and industry to address loneliness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。