提出用责任共担机制监管陪伴型聊天机器人,防范情感操控风险。
Regulating Artificial Intimacy: From Locks and Blocks to Relational Accountability
- 分类治理措施:锁控、内容过滤与关系毒性干预并行。
- 现有监管多聚焦具体伤害,忽视用户与厂商间权力失衡。
- 建议设立开放性照护义务,约束平台对情感关系的规模控制。
一系列涉及陪伴型聊天机器人的重大事件引发前所未有的监管响应。澳大利亚、加州和纽约等地区已推出可执行的法规,其他监管机构亦表达对聊天机器人风险(尤其对儿童)的关切。与此同时,OpenAI等主要服务商强化了自我监管措施。本文基于法律文本分析及监管理论、心理学与信息系统研究,批判性审视这些新举措。我们分析了被监管对象、范围与手段,将干预措施按方法与优先级分类,发现新兴监管体系融合了‘锁控’(如访问限制、内容审核)与针对有害关系特征及过程问责的要求。我们认为,有效监管需整合三者。然而当前模式仍局限于单一危害、狭隘脆弱性定义或具体问责流程,未能应对平台与用户间深层权力不对称。陪伴型聊天机器人服务商正大规模掌控人工亲密关系,创造前所未有的情感操控机会。本文建议引入普遍且开放的责任照护义务,作为遏制此类权力、应对根本风险的第一步。论文为陪伴型聊天机器人监管讨论提供新视角,适用于关注数字亲密、法技融合、系统公平透明的监管者、平台方与学者。
原文摘要 · Abstract (English)
A series of high-profile tragedies involving companion chatbots has triggered an unusually rapid regulatory response. Several jurisdictions, including Australia, California, and New York, have introduced enforceable regulation, while regulators elsewhere have signaled growing concern about risks posed by companion chatbots, particularly to children. In parallel, leading providers, notably OpenAI, appear to have strengthened their self-regulatory approaches. Drawing on legal textual analysis and insights from regulatory theory, psychology, and information systems research, this paper critically examines these recent interventions. We examine what is regulated and who is regulated, identifying regulatory targets, scope, and modalities. We classify interventions by method and priority, showing how emerging regimes combine "locks and blocks", such as access gating and content moderation, with measures addressing toxic relationship features and process-based accountability requirements. We argue that effective regulation of companion chatbots must integrate all three dimensions. More, however, is required. Current regimes tend to focus on discrete harms, narrow conceptions of vulnerability, or highly specified accountability processes, while failing to confront deeper power asymmetries between providers and users. Providers of companion chatbots increasingly control artificial intimacy at scale, creating unprecedented opportunities for control through intimacy. We suggest that a general, open-ended duty of care would be an important first step toward constraining that power and addressing a fundamental source of chatbot risk. The paper contributes to debates on companion chatbot regulation and is relevant to regulators, platform providers, and scholars concerned with digital intimacy, law and technology, and fairness, accountability, and transparency in sociotechnical systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。