用反事实检测和减轻大模型幻觉，无需重训练。
Counterfactual Probing for Hallucination Detection and Mitigation in Large Language Models
- 生成看似合理却含小错误的反事实语句，测试模型反应。
- 在TruthfulQA等数据集上，幻觉检测效果优于基线方法。
- 可实时集成，适合需要高可信输出的应用场景。
大语言模型在多项任务中表现出色，但常生成流畅却事实错误的幻觉内容。本文提出反事实探测方法，动态生成看似合理但含细微事实错误的反事实陈述，并评估模型对这些扰动的敏感性。我们假设真实知识对反事实变化具有鲁棒性，而幻觉内容在面对合理替代方案时表现出不一致的信心模式。在TruthfulQA、事实陈述数据集及精心收集的幻觉样本上的综合评估表明，反事实探测在检测性能上优于基线方法；结合自适应缓解策略，平均将幻觉得分降低24.5%。该方法无需模型重训练，可作为实时验证机制嵌入现有大模型流水线。
原文摘要 · Abstract (English)
Large Language Models have demonstrated remarkable capabilities across diverse tasks, yet they frequently generate hallucinations outputs that are fluent but factually incorrect or unsupported. We propose Counterfactual Probing, a novel approach for detecting and mitigating hallucinations in LLM outputs. Our method dynamically generates counterfactual statements that appear plausible but contain subtle factual errors, then evaluates the model's sensitivity to these perturbations. We hypothesize that genuine knowledge exhibits robustness to counterfactual variations, while hallucinated content shows inconsistent confidence patterns when confronted with plausible alternatives. Our comprehensive evaluation on TruthfulQA, factual statement datasets, and curated hallucination examples demonstrates that counterfactual probing achieves superior detection performance compared to baseline methods, while our adaptive mitigation strategies reduce hallucination scores by an average of 24.5%. The approach requires no model retraining and can be integrated into existing LLM pipelines as a realtime verification mechanism.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。