通过游戏实验发现，中学生能逐步识别大模型伪装者。
Detecting AI Impostors: How Do Middle Schoolers Identify LLM Agents in a Live Collaborative Setting?
- 设计游戏DoppelBot模拟社交欺骗场景
- 检测准确率随时间提升，依赖社交线索而非语言特征
- 适合研究青少年AI认知与数字素养的教育者
大型语言模型可模仿人类写作风格，引发社交场景中的冒充、信任与检测问题。这一问题对频繁使用生成式AI的青少年尤为关键。本文提出名为DoppelBot的合作社交推理游戏，用于研究青少年如何识别和应对AI冒充。通过针对中学生的多轮实验，探究了游戏是否引发对隐私与冒充的反思、重复接触如何影响检测准确率（尤其当代理更加个性化时），以及学生使用的识别策略。结果显示，学生检测准确率随时间提升，其判断从依赖语言特征转向利用共享的社会与情境信号；同时表现出对AI局限性（如具身性）的理解，并思考数据隐私等深层议题。为支持后续研究，作者发布了匿名化的游戏对话记录与投票行为数据集。
原文摘要 · Abstract (English)
LLMs can imitate how people write, which raises concerns about impersonation, trust, and detection in social settings. These concerns are especially important for adolescents, who use generative AI frequently but may struggle to recognize it. We introduce \textit{DoppelBot}, a cooperative social deduction game designed to study how young people detect and respond to AI impersonation. Through studies with middle schoolers, we investigate whether a DoppelBot prompts reflection on privacy and impersonation, how repeated exposure affects AI-detection accuracy as agents become more personalized, and which strategies students use to identify AI doppelgängers. We find that students' detection accuracy improves over time, driven by a shift from relying on linguistic cues to leveraging shared social and contextual signals. Students also demonstrated an understanding of AI limitations such as embodiment and reflected on broader issues such as data privacy. To support future research, we release an anonymized dataset of game transcripts and voting behavior.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。