用40亿参数小模型帮大模型查谎，提升问答可信度。
HalluGuard: Evidence-Grounded Small Reasoning Models to Mitigate Hallucinations in Retrieval-Augmented Generation
- 构建合成数据集并用偏好优化训练小模型，实现判别式纠错
- 在RAGTruth上达84.0%准确率，仅用一半参数媲美大模型
- 适合需要轻量、可信生成的落地场景，如医疗、金融问答
大语言模型虽在众多自然语言任务中表现优异，但仍易产生幻觉，限制其在真实应用中的可信度。本文提出HalluGuard，一个40亿参数的小型推理模型（SRM），用于缓解检索增强生成（RAG）中的幻觉问题。该模型可判断文档-主张对是否具有证据支持，并生成基于证据的解释以增强透明性。方法结合：(i) 基于FineWeb构建的领域无关合成数据集，经多阶段清洗与重构；(ii) 合成的有据主张与幻觉主张；(iii) 基于几率比偏好优化的偏好微调，将大模型推理能力压缩至小型骨干网络。在LLM-AggreFact基准的RAGTruth子集上，HalluGuard达到84.0%的平衡准确率（BAcc），与专用模型MiniCheck（7B；84.0%）和Granite Guardian 3.3（8B；82.2%）相当，参数量约为其一半。在完整基准上达75.7% BAcc，与GPT-4o（75.9%）相当。论文将在接受后以Apache 2.0协议发布HalluGuard及数据集。
原文摘要 · Abstract (English)
Large Language Models (LLMs) excel in many NLP tasks but remain prone to hallucinations, limiting trust in real-world applications. We present HalluGuard, a 4B-parameter Small Reasoning Model (SRM) for mitigating hallucinations in Retrieval-Augmented Generation (RAG). HalluGuard classifies document-claim pairs as grounded or hallucinated and produces evidence-grounded justifications for transparency. Our approach combines (i) a domain-agnostic synthetic dataset derived from FineWeb and refined through multi-stage curation and data reformation, (ii) synthetic grounded and hallucinated claims, and (iii) preference-based fine-tuning with Odds Ratio Preference Optimization to distill large-model reasoning into a smaller backbone. On the RAGTruth subset of the LLM-AggreFact benchmark, HalluGuard achieves 84.0% balanced accuracy (BAcc), rivaling specialized models, MiniCheck (7B; 84.0%) and Granite Guardian 3.3 (8B; 82.2%) while using roughly half their parameters. Over the full benchmark it reaches 75.7% BAcc, matching larger general-purpose LLMs such as GPT-4o (75.9%). We will release HalluGuard and datasets under Apache 2.0 upon acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。