用几何角度度量RAG幻觉，发现回答常绕着问题转而不靠近上下文。
Semantic Grounding Index: Geometric Bounds on Context Engagement in RAG Systems
- 通过问答与上下文在超球面的夹角比，定义可量化幻觉的语义锚定指数。
- 在HaluEval上效果显著，最大效应量达1.28，且随问题-上下文距离增大而提升。
- 适合用于生产环境中的幻觉检测，可作为可信度评分而非仅排序依据。
当检索增强生成（RAG）系统产生幻觉时，嵌入空间中会留下何种几何痕迹？本文提出语义锚定指数（SGI），即响应与问题、上下文在单位超球面 $\mathbb{S}^{d-1}$ 上的夹角之比。核心发现为‘语义懒惰’：幻觉响应仍保持与问题的角距离近，而非向检索到的上下文靠近。在 HaluEval 数据集（n=5,000）上，五种嵌入模型均显示大效应量（Cohen's $d$ 范围 0.92 至 1.28），跨模型相关性均值 $r$=0.85。理论上，基于球面三角不等式，SGI 的判别能力应随问题-上下文夹角 $θ(q,c)$ 增大而增强，实证验证了这一趋势：效应量从低 $θ$ 时的 $d$=0.61 升至高 $θ$ 时的 $d$=1.27，AUC 从 0.72 提升至 0.83。子组分析表明，SGI 在长回答（$d$=2.05）和短问题（$d$=1.22）上表现更优，且对上下文长度鲁棒。校准分析显示 ECE=0.10，说明 SGI 可作为概率估计而非仅排名。在 TruthfulQA 上的负结果（AUC=0.478）表明，该指标衡量的是话题参与度而非事实准确性。SGI 为生产级 RAG 系统提供高效、理论严谨的幻觉识别基础设施。
原文摘要 · Abstract (English)
When retrieval-augmented generation (RAG) systems hallucinate, what geometric trace does this leave in embedding space? We introduce the Semantic Grounding Index (SGI), defined as the ratio of angular distances from the response to the question versus the context on the unit hypersphere $\mathbb{S}^{d-1}$.Our central finding is \emph{semantic laziness}: hallucinated responses remain angularly proximate to questions rather than departing toward retrieved contexts. On HaluEval ($n$=5,000), we observe large effect sizes (Cohen's $d$ ranging from 0.92 to 1.28) across five embedding models with mean cross-model correlation $r$=0.85. Crucially, we derive from the spherical triangle inequality that SGI's discriminative power should increase with question-context angular separation $θ(q,c)$-a theoretical prediction confirmed empirically: effect size rises monotonically from $d$=0.61 -low $θ(q,c)$, to $d$=1.27 -high $θ(q,c)$, with AUC improving from 0.72 to 0.83. Subgroup analysis reveals that SGI excels on long responses ($d$=2.05) and short questions ($d$=1.22), while remaining robust across context lengths. Calibration analysis yields ECE=0.10, indicating SGI scores can serve as probability estimates, not merely rankings. A critical negative result on TruthfulQA (AUC=0.478) establishes that angular geometry measures topical engagement rather than factual accuracy. SGI provides computationally efficient, theoretically grounded infrastructure for identifying responses that warrant verification in production RAG deployments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。