通过分析模型各层词元逻辑值，精准识别幻觉内容。
CHAIR -- Classifier of Hallucination as Improver
- 提取每层词元逻辑值的极值、均值等特征，构建紧凑检测信号。
- 在TruthfulQA和MMLU上零样本检测准确率显著提升。
- 为改进生成质量提供新思路，适合关注模型可信度的研究者。
本文提出CHAIR（幻觉分类器作为优化器），一种基于监督学习的幻觉检测框架，通过分析每个词元在模型各层内部逻辑值来实现检测。该方法提取跨所有层的词元逻辑值中的最大值、最小值、均值、标准差及斜率等紧凑特征，有效避免过拟合。在TruthfulQA与MMLU数据集上的实验表明，CHAIR在零样本场景下检测准确率显著提升，展现出强鲁棒性与泛化能力。除幻觉检测外，该方法还揭示了利用内部表示设计先进解码策略的潜力。通过挖掘逻辑值中的模式，可进一步发展更复杂的模型与自适应解码方法，以减少幻觉并提升文本生成质量。CHAIR不仅提供实用的幻觉检测方案，也为探索大语言模型中更丰富的表征以增强事实性与连贯性奠定基础。
原文摘要 · Abstract (English)
In this work, we introduce CHAIR (Classifier of Hallucination As ImproveR), a supervised framework for detecting hallucinations by analyzing internal logits from each layer of every token. Our method extracts a compact set of features such as maximum, minimum, mean, standard deviation, and slope-from the token logits across all layers, enabling effective hallucination detection without overfitting. Experiments on TruthfulQA and MMLU datasets demonstrate that CHAIR significantly improves detection accuracy, particularly in zero-shot scenarios, showcasing its robustness and generalizability. Beyond hallucination detection, CHAIR highlights the potential of using internal representations for designing advanced decoding strategies. By leveraging patterns in logits, we suggest that more sophisticated models and adaptive decoding methods could further reduce hallucinations and enhance text completion quality. CHAIR not only offers a practical solution for detecting hallucinations but also lays the groundwork for exploring richer representations in LLMs to improve their factuality and coherence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。