为多智能体大模型设计可区分证据类型的幻觉检测与恢复框架
GSAR: Typed Grounding for Hallucination Detection and Recovery in Multi-Agent LLMs
- 将声明分为四类：有据、无据、矛盾、互补，区分不同证据价值
- 按证据类型赋权并计算惩罚矛盾的加权可信度得分
- 在限定计算预算下分三步决策：继续、重生成或重规划
自主多智能体大模型系统被广泛用于调查运营事件并生成结构化诊断报告。其可信度取决于每个陈述是否基于观测证据而非模型内部推断。现有评估方法（二分类器、大模型作为裁判、自修正循环）将支持证据视为可互换，仅输出单一信号，无法对下游行为进行原则性控制。本文提出GSAR，一种融合证据类型评分与分层恢复的接地性评估与重规划框架：(i) 将声明划分为四类（有据、无据、矛盾、互补），赋予非冗余替代视角优先地位；(ii) 为不同类型证据分配反映认知强度的权重；(iii) 计算考虑不对称矛盾惩罚的加权接地度分数；(iv) 将该分数耦合至三阶决策函数（继续、重生成、重规划），在显式计算预算下驱动有限迭代外循环。我们形式化算法，证明六个结构性质，并在包含金标准维基百科证据的FEVER数据集上，通过四个独立训练的大模型裁判（gpt-5.4, claude-sonnet-4-6, claude-opus-4-7, gemini-2.5-pro）验证五个设计假设。所有消融实验均在各裁判上呈现相同方向：对rho=0的95%置信区间排除0值；在Opus 4.7上，无互补项消融的置信区间为[-96,-68]，样本量n=1000时，三个独立裁判收敛于DeltaS(rho=0)=+0.058。还包含与Vectara HHEM-2.1-Open的直接对比。据我们所知，GSAR是首个将证据类型评分与受限计算下的分层恢复结合的公开接地性框架。
原文摘要 · Abstract (English)
Autonomous multi-agent LLM systems are increasingly deployed to investigate operational incidents and produce structured diagnostic reports. Their trustworthiness hinges on whether each claim is grounded in observed evidence rather than model-internal inference. Existing groundedness evaluators (binary classifiers, LLM-as-judge scalars, self-correction loops) treat supporting evidence as interchangeable and emit a single signal that offers no principled control over downstream action. We present GSAR, a grounding-evaluation and replanning framework that (i) partitions claims into a four-way typology (grounded, ungrounded, contradicted, complementary), giving first-class standing to non-redundant alternative perspectives; (ii) assigns evidence-type-specific weights reflecting epistemic strength; (iii) computes an asymmetric contradiction-penalised weighted groundedness score; and (iv) couples that score to a three-tier decision function (proceed, regenerate, replan) driving a bounded-iteration outer loop under an explicit compute budget. We formalise the algorithm, prove six structural properties, and evaluate five design claims on FEVER with gold Wikipedia evidence under four independently-trained LLM judges (gpt-5.4, claude-sonnet-4-6, claude-opus-4-7, gemini-2.5-pro). Every ablation reproduces in the same direction on every judge: bootstrap 95% CIs on the rho=0 effect exclude 0 on all four; the no-complementary ablation under Opus 4.7 has CI [-96,-68] of 200; at n=1000 three independent judges converge to DeltaS(rho=0)=+0.058. A head-to-head against Vectara HHEM-2.1-Open is included. To our knowledge, GSAR is the first published groundedness framework coupling evidence-typed scoring with tiered recovery under an explicit compute budget.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。