arXiv:2603.10060cs.CRcs.AI2026-03被引 1

用可验证的执行凭证检测AI代理幻觉,实时高效且易用。

Tool Receipts, Not Zero-Knowledge Proofs: Practical Hallucination Detection for AI Agents

  • 为每个回答声明标注知识来源,通过签名凭证交叉验证
  • 94.2%检出虚假工具调用,15ms内完成每条响应验证
  • 适合需要快速反馈的交互式AI应用,提供可理解的信任信号

通过工具调用执行任务的AI代理常产生幻觉——虚构工具调用、错误陈述输出数量或将推断当作事实。现有可验证推理方法依赖零知识证明,虽具密码学保障但每查询需数分钟,不适用于交互式代理。本文提出NabaOS,一种受印度认识论(Nyaya Shastra)启发的轻量级验证框架,将大模型响应中的每一项声明按其认知来源(pramana)分类:直接工具输出(pratyaksha)、推断(anumana)、外部证词(shabda)、缺失(abhava)或无根据意见。系统生成由HMAC签名的工具执行凭证,大模型无法伪造,并实时交叉核对声明与凭证以检测幻觉。在包含1800个场景、六种幻觉类型、四种语言的NyayaVerifyBench上评估,NabaOS检出94.2%的虚构工具引用、87.6%的计数错误和91.3%的虚假缺失声明,单次响应验证开销低于15ms。对于多步网页任务,交叉检查协议通过独立重取捕获78.4%的网址伪造。相较zkLLM(180秒/查询)、TOPLOC、SPEX、张量承诺及自洽性检查,NabaOS在成本-延迟-覆盖度之间取得最优平衡:94.2%覆盖率仅耗时<15毫秒,而zkLLM虽近乎完美覆盖却需180,000毫秒。对于交互式代理,基于凭证的实用验证优于密码学证明,且认知分类提供比二元判断更有价值的信任信息。

原文摘要 · Abstract (English)

AI agents that execute tasks via tool calls frequently hallucinate results - fabricating tool executions, misstating output counts, or presenting inferences as facts. Recent approaches to verifiable AI inference rely on zero-knowledge proofs, which provide cryptographic guarantees but impose minutes of proving time per query, making them impractical for interactive agents. We propose NabaOS, a lightweight verification framework inspired by Indian epistemology (Nyaya Shastra), which classifies every claim in an LLM response by its epistemic source (pramana): direct tool output (pratyaksha), inference (anumana), external testimony (shabda), absence (abhava), or ungrounded opinion. Our runtime generates HMAC-signed tool execution receipts that the LLM cannot forge, then cross-references claims against these receipts to detect hallucinations in real time. We evaluate on NyayaVerifyBench, a new benchmark of 1,800 agent response scenarios across four languages with injected hallucinations of six types. NabaOS detects 94.2% of fabricated tool references, 87.6% of count misstatements, and 91.3% of false absence claims, with <15ms verification overhead per response. For deep delegation (agents performing multi-step web tasks), our cross-checking protocol catches 78.4% of URL fabrications via independent re-fetching. We compare against five approaches: zkLLM (cryptographic proofs, 180s/query), TOPLOC (locality-sensitive hashing), SPEX (sampling-based proof of execution), tensor commitments, and self-consistency checking. NabaOS achieves the best cost-latency-coverage trade-off for interactive agents: 94.2% coverage at <15ms versus zkLLM's near-perfect coverage at 180,000ms. For interactive agents, practical receipt-based verification provides better cost-benefit than cryptographic proofs, and epistemic classification gives users actionable trust signals rather than binary judgments.

幻觉检测AI代理可信推理轻量验证

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。