通过多轮生成分析，用31维特征检测大模型幻觉，无需内部信息。
BEACON: Behavioral Entropy Aggregation for Cross-Model Hallucination Detection in Large Language Models
- 基于多轮生成输出，融合语义熵、推理一致性等信号构建特征向量。
- 在7个基准上达到0.8123的AUROC，比现有方法提升超24个百分点。
- 仅需5次调用即可部署，适合黑盒API场景下的幻觉检测。
大语言模型中的幻觉（即生成事实错误或无依据内容）仍是可靠部署的关键障碍。本文提出BEACON（Behavioral Entropy Aggregation for Cross-model hallucination detectiON），一种纯基于模型输出的黑盒幻觉检测框架，无需访问内部表示或外部知识库。BEACON从结构化多轮生成中提取31维特征向量，整合基于NLI的语义熵、嵌入几何、思维链一致性及改写稳定性信号。在7个基准上使用7,617个标注样本训练的梯度提升分类器，获得0.8123 ± 0.0102的AUROC（95%置信区间：0.7632-0.8251），显著优于单一语义熵（+0.2298）和SelfCheckGPT类一致性基线（+0.2457）。特征重要性分析表明，幻觉具有多维度本质，需综合不确定性信号。一个高效的5次调用变体实现0.7795 AUROC，支持在黑盒LLM API上的实际部署。
原文摘要 · Abstract (English)
Hallucination in large language models (LLMs), defined as the generation of factually incorrect or unsupported content, remains a critical barrier to reliable deployment. We present BEACON (Behavioral Entropy Aggregation for Cross-model hallucination detectiON), a black-box hallucination detection framework that operates purely on model outputs without requiring access to internal representations or external knowledge bases. BEACON extracts a 31-dimensional feature vector from structured multi-pass generation, integrating NLI-based semantic entropy, embedding geometry, chain-of-thought consistency, and paraphrase stability signals. A gradient-boosted classifier trained on 7,617 labeled examples across seven benchmarks achieves 0.8123 +/- 0.0102 AUROC (95% CI: 0.7632-0.8251), outperforming standalone semantic entropy (+0.2298) and SelfCheckGPT-style consistency baselines (+0.2457). Feature importance analysis shows that hallucination is inherently multi-dimensional, requiring combined uncertainty signals. An efficient 5-call variant achieves 0.7795 AUROC, enabling practical deployment across black-box LLM APIs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。