基于真实用户心理体验，梳理出AI对话系统19类风险行为与21种心理影响。
From Lived Experience to Insight: Unpacking the Psychological Risks of Using AI Conversational Agents
- 通过283人调查和专家工作坊，从真实体验中提炼心理风险
- 构建包含19种行为、21种心理影响的完整风险分类体系
- 提出多路径情景框架，适合设计安全可靠的AI助手
近年来，人工智能对话代理因提升效率与支持心理健康而日益普及。然而，以往研究多未充分捕捉用户的真实体验，且心理风险常被归入更广泛的AI风险范畴，导致其影响被低估。为此，本文基于283名有心理疾病经历者的调查与专家工作坊，提出一个聚焦心理风险的新风险分类体系。该体系包含19种AI行为、21种负面心理影响及15种使用情境，并引入一种新型多路径情景分析框架，揭示AI行为、心理影响与用户背景间的复杂互动关系。根据工作坊反馈，进一步提出面向开发者、研究者与政策制定者的具体设计建议。本研究深化了对AI对话系统心理风险的理解，提供了可落地的实践指导。
原文摘要 · Abstract (English)
Recent gains in popularity of AI conversational agents have led to their increased use for improving productivity and supporting well-being. While previous research has aimed to understand the risks associated with interactions with AI conversational agents, these studies often fall short in capturing the lived experiences of individuals. Additionally, psychological risks have often been presented as a sub-category within broader AI-related risks in past taxonomy works, leading to under-representation of the impact of psychological risks of AI use. To address these challenges, our work presents a novel risk taxonomy focusing on psychological risks of using AI gathered through the lived experiences of individuals. We employed a mixed-method approach, involving a comprehensive survey with 283 people with lived mental health experience and workshops involving experts with lived experience to develop a psychological risk taxonomy. Our taxonomy features 19 AI behaviors, 21 negative psychological impacts, and 15 contexts related to individuals. Additionally, we propose a novel multi-path vignette-based framework for understanding the complex interplay between AI behaviors, psychological impacts, and individual user contexts. Finally, based on the feedback obtained from the workshop sessions, we present design recommendations for developing safer and more robust AI agents. Our work offers an in-depth understanding of the psychological risks associated with AI conversational agents and provides actionable recommendations for policymakers, researchers, and developers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。