通过融合语义与词元信号,提升大模型幻觉检测准确率。
From Tokens to Semantics: Leveraging Complementary Signals for Hallucination Detection in Black-Box LLMs

- 结合语义分歧和词元不确定性,设计混合检测方法。
- 在多个数据集上,监督方法在近半数场景表现最优。
- 无需标注数据的方法也有效,但需精细调参。
当大语言模型用于公开或高风险任务时,遗漏的虚假内容可能损害用户与机构,而误报则浪费有限的人工审核资源。在无可信上下文或参考文档的情况下,本文研究可通过黑盒API获取的两种信号:语义熵(衡量采样响应意义的分歧程度)和基于词元概率的不确定性。二者失效模式互补:语义熵在响应集中于单一语义簇时失效,而词元不确定性可能忽略一致自信的错误。本文提出改进的词元不确定性聚合方法TopK,评估混合方法CoCoA(结合目标响应不确定性与语义差异),并提出两种监督方法:Gated将单簇情况路由至聚合词元特征分类器,Stacked联合学习语义不确定性和更广词元特征。在七个基准上评估,包含五个公开基准(四文本数据集及多模态手写支票提取)和两个构造基准(金融摘要、长文本问答),使用四种语言模型。跨模型与数据集评估中,Stacked在近一半情况下表现最佳;而TopK与CoCoA无需监督标签,性能仍具竞争力,但阈值需谨慎校准。无方法在所有场景均最优。因此,我们在1%至15%假阳性率预算下评估性能,分析生成与校准选择的敏感性,并考察数据集特性差异的影响。
原文摘要 · Abstract (English)
When LLMs support public-facing or high-stakes workflows, missed fabrications can harm users and institutions, while false alarms consume limited human-review capacity. When no trusted context or reference document is available, we study two signals accessible through black-box model APIs: semantic entropy, which measures disagreement among sampled response meanings, and uncertainty derived from token log-probabilities. Their failure modes can be complementary: semantic entropy becomes uninformative when responses form one semantic cluster, while token uncertainty can miss consistently confident errors. We extend token-based uncertainty detection by aggregating token-level signals across sampled responses through our TopK method, evaluate the hybrid CoCoA method, which combines target-response uncertainty with semantic dissimilarity, and propose and study two supervised methods: Gated, which routes single-cluster cases to an aggregated-token-feature classifier, and Stacked, which learns jointly from semantic uncertainty and broader token features. We evaluate seven benchmarks, including five public benchmarks (four text datasets and multimodal handwritten-cheque extraction) and two constructed benchmarks (Financial Summaries and Long-Text QA), using four language models. In our evaluation across models and datasets, Stacked gave the best performance in nearly half of the cases, while TopK and CoCoA remain competitive without supervised training labels, although their thresholds require careful calibration. No method is universally strongest. We therefore evaluate performance at false-positive-rate budgets from 1% to 15%, assess their sensitivity to generation and calibration choices, and examine variation across dataset characteristics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。