arXiv:2607.17883cs.CLcs.AI2026-07

构建六层防御体系,让企业AI系统可靠生成不虚构内容。

Zero Hallucination, by Construction: Hallucination-Aware Layered Oversight for Trustworthy Enterprise AI

  • 用六层结构化设计,把幻觉当可管控的故障处理。
  • 实测在理赔信息提取任务中显著降低虚构内容,提升可信度。
  • 适合需要高可靠性、强合规性的企业级AI部署场景。

企业不会部署不可信的AI代理,而最被诟病的原因是幻觉:模型输出看似自信流畅,实则虚假。通常做法是等待无幻觉模型,但我们认为这是错误目标。大语言模型本质具备生成无依据文本的能力,规模越大越难消除。在原始模型上加一个判断器只能捕捉部分错误,即便精心设计的检索流程也可能伪造引用。我们重新定义目标:‘零幻觉’不是模型自带属性,而是系统强制的特性。提出HALO(幻觉感知分层监督)架构,将幻觉视为可管控的失败模式而非必须根除的问题。该架构包含六层防护:基于检索和审批内容的生成;受限且确定性的执行以限制出错范围;多信号验证(使用大模型判别器与源文本证据比对)评估每条输出的可信度;置信度不足时主动拒绝回答;全程可追溯所有检索、工具调用与生成记录;持续监控机制实时检测偏差、触发警报,并通过再生与统计验证闭环优化代理。详细阐述各层设计,特别强调基于证据的置信度(直接核对抽取结果与原文,而非依赖模型自我宣称)。在受监管的理赔信息提取任务中进行验证。

原文摘要 · Abstract (English)

Enterprises will not deploy AI agents they cannot trust, and the most-cited reason for distrust is hallucination: confident, fluent output that is simply not true. The common response is to wait for a model that does not hallucinate. We argue that this is the wrong target. Large language models are, by construction, capable of generating unsupported text, and no amount of scale removes the possibility; a faithfulness judge bolted onto a raw model catches some errors but still ships others, and even well-curated retrieval pipelines have been shown to fabricate citations. We reframe the goal: "zero hallucination" is not a property a model possesses but a property a system enforces. We present HALO (Hallucination-Aware Layered Oversight), an assurance architecture which treats hallucination as a containable failure mode rather than an eliminable one. HALO composes six layers of defense: grounded generation over retrieved, approved content; constrained, deterministic execution that bounds where the model can err; multi-signal verification that scores every output for groundedness and hallucination using both an LLM judge and evidence-based checks against the source text; calibrated abstention, so the system declines rather than guesses when grounding is insufficient; total traceability of every retrieval, tool call, and generation; and continuous oversight that detects drift, alerts on threshold breaches, and closes the loop by regenerating and statistically validating improved agents. We detail each layer, give particular attention to evidence-based confidence (which verifies extractions against the source document rather than trusting the model's self-reported certainty), and illustrate the architecture on a regulated claims-extraction workload.

企业AI幻觉抑制系统架构可信生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。