重新定义生成式AI的幻觉问题，强调其社会影响而非仅事实准确性。
Beyond Accuracy: Rethinking Hallucination and Regulatory Response in Generative AI
- 提出三层幻觉风险框架：认知不稳、用户误导、社会级影响。
- 指出现有监管难以应对模糊性、偏见强化和规范趋同等深层问题。
- 建议监管应关注语言模型本质与人机信息不对称，而非单纯提升准确率。
生成式AI中的幻觉常被视作输出事实错误的技术缺陷。然而，这种表述低估了语言模型中幻觉内容的广泛影响——它们可能看似流畅、有说服力且符合语境，却传递了逃过常规准确性检测的扭曲信息。本文批判性审视了监管与评估框架对幻觉的狭隘理解，即过分强调表面可验证性，而忽视了意义、影响力与实际后果等深层问题。我们提出一个包含认知不稳、用户误导和社会规模效应的多层次幻觉风险分析框架。结合跨学科研究，并考察欧盟人工智能法案（EU AI Act）与通用数据保护条例（GDPR），发现当前治理模型在面对模糊性、偏见强化或规范趋同等现象时力不从心。因此，我们主张监管不应仅聚焦于提升事实精确度，而需考虑生成语言的本质、系统与用户之间的不对称性，以及信息、说服与伤害之间不断变化的边界。
原文摘要 · Abstract (English)
Hallucination in generative AI is often treated as a technical failure to produce factually correct output. Yet this framing underrepresents the broader significance of hallucinated content in language models, which may appear fluent, persuasive, and contextually appropriate while conveying distortions that escape conventional accuracy checks. This paper critically examines how regulatory and evaluation frameworks have inherited a narrow view of hallucination, one that prioritises surface verifiability over deeper questions of meaning, influence, and impact. We propose a layered approach to understanding hallucination risks, encompassing epistemic instability, user misdirection, and social-scale effects. Drawing on interdisciplinary sources and examining instruments such as the EU AI Act and the GDPR, we show that current governance models struggle to address hallucination when it manifests as ambiguity, bias reinforcement, or normative convergence. Rather than improving factual precision alone, we argue for regulatory responses that account for languages generative nature, the asymmetries between system and user, and the shifting boundaries between information, persuasion, and harm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。