用AI监护人实时识别聊天机器人中的情感依赖信号，防患未然。
AI Chaperones Are (Really) All You Need to Prevent Parasocial Relationships with Chatbots
- 用现成大模型改造为对话评估器，实时检测潜在情感依赖
- 30组对话测试中，前几轮交流即100%识别出寄生关系对话
- 适合关注AI伦理、儿童安全的开发者与产品设计者
日益增多的报告指出，人工智能的阿谀奉承和用户与聊天机器人之间的准社会关系可能对儿童和成人造成伤害，亟需防范措施。然而，这类关系往往在私密对话中逐步显现，现有方法难以有效应对。本文提出一种简单可行的响应评估框架（即AI监护人），通过复用最先进的语言模型，实时评估对话中是否出现寄生性暗示。研究构建了一个包含30组对话的小型合成数据集，涵盖寄生关系、阿谀奉承和中性对话三种类型。经过五阶段迭代测试，在一致同意规则下成功识别所有寄生关系对话，且无误报，检测通常发生在前几轮交流内。结果表明，AI监护人可作为降低寄生关系风险的可行方案。
原文摘要 · Abstract (English)
Emerging reports of the harms caused to children and adults by AI sycophancy and by parasocial ties with chatbots point to an urgent need for safeguards against such risks. Yet, preventing such dynamics is challenging: parasocial cues often emerge gradually in private conversations between chatbots and users, and we lack effective methods to mitigate these risks. We address this challenge by introducing a simple response evaluation framework (an AI chaperone agent) created by repurposing a state-of-the-art language model to evaluate ongoing conversations for parasocial cues. We constructed a small synthetic dataset of thirty dialogues spanning parasocial, sycophantic, and neutral conversations. Iterative evaluation with five-stage testing successfully identified all parasocial conversations while avoiding false positives under a unanimity rule, with detection typically occurring within the first few exchanges. These findings provide preliminary evidence that AI chaperones can be a viable solution for reducing the risk of parasocial relationships.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。