提出细粒度幻觉检测与修正方法，提升大模型解题可靠性
FG-PRM: Fine-grained Hallucination Detection and Mitigation in Language Model Mathematical Reasoning
- 构建六类数学推理幻觉分类体系，实现步骤级幻觉识别
- 在GSM8K和MATH上显著提升模型准确率，验证方法有效性
- 自动生成标注数据，解决人工标注成本高难题
大语言模型在复杂多步推理任务中存在幻觉问题，尤其在数学求解中尤为突出。现有方法仅能检测幻觉存在，缺乏对类型和表现形式的精细理解。本文首先建立涵盖六类常见数学推理幻觉的分类体系，进而提出FG-PRM（细粒度过程奖励模型），实现步骤级的幻觉检测与缓解。为克服人工标注数据的局限性，提出基于LLM的自动化细粒度幻觉数据生成方法。实验表明，FG-PRM在两项核心任务上表现优异：1）细粒度幻觉检测——对每个推理步骤进行幻觉类型分类；2）验证任务——对多个模型输出进行排序以选择最准确解法。在GSM8K和MATH基准测试中，该方法显著提升大模型性能，证明细粒度监督可有效增强大模型推理的可靠性和可解释性。
原文摘要 · Abstract (English)
Hallucinations in large language models (LLMs) pose significant challenges in tasks requiring complex multi-step reasoning, such as mathematical problem-solving. Existing approaches primarily detect the presence of hallucinations but lack a nuanced understanding of their types and manifestations. In this paper, we first introduce a comprehensive taxonomy that categorizes the common hallucinations in mathematical reasoning tasks into six types. We then propose FG-PRM (Fine-Grained Process Reward Model), an augmented model designed to detect and mitigate hallucinations in a fine-grained, step-level manner. To address the limitations of manually labeling training data, we propose an automated method for generating fine-grained hallucination data using LLMs. Our FG-PRM demonstrates superior performance across two key tasks: 1) Fine-grained hallucination detection: classifying hallucination types for each reasoning step; and 2) Verification: ranking multiple LLM-generated outputs to select the most accurate solution. Our experiments show that FG-PRM excels in fine-grained hallucination detection and substantially boosts the performance of LLMs on GSM8K and MATH benchmarks. These results highlight the benefits of fine-grained supervision in enhancing the reliability and interpretability of LLM reasoning processes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。