arXiv:2502.11959cs.AI2025-02中稿 · Machine Intelligen…被引 4

通过结构化推理提升事实验证的自改进能力,解决错误推理循环问题。

STRIVE: Structured Reasoning for Self-Improvement in Claim Verification

  • 设计分解、实体分析与证据验证三步结构化推理流程
  • 在HOVER数据集上比基线模型提升31.4%,较CoT提升20.7%
  • 适合需要高可靠事实验证的应用场景

事实验证旨在判断陈述是否得到证据支持。自改进方法通过生成推理链并选取正确结果进行训练,在数学求解等任务中表现良好。但在事实验证中,低质量推理链可能错误匹配二元真值标签,将错误推理引入训练过程,导致性能下降。为此,我们提出STRIVE:结构化推理用于自改进的事实验证。该方法引入包含陈述分解、实体分析和证据锚定验证的结构化推理设计,提升推理质量、减少错误,并为自改进提供额外监督信号。STRIVE首先通过少量标注样本微调基础模型以学习结构化推理模式;随后生成所有训练样本的推理链,仅选择正确且结构合理的链用于后续自改进训练。实验表明,STRIVE在HOVER数据集上相较基线模型提升31.4%,较Chain of Thought提升20.7%,证明其有效性。

原文摘要 · Abstract (English)

Claim verification is the task of determining whether a claim is supported or refuted by evidence. Self-improvement methods, where reasoning chains are generated and those leading to correct results are selected for training, have succeeded in tasks like mathematical problem solving. However, in claim verification, this approach struggles. Low-quality reasoning chains may falsely match binary truth labels, introducing faulty reasoning into the self-improvement process and ultimately degrading performance. To address this, we propose STRIVE: Structured Reasoning for Self-Improved Verification. Our method introduces a structured reasoning design with Claim Decomposition, Entity Analysis, and Evidence Grounding Verification. These components improve reasoning quality, reduce errors, and provide additional supervision signals for self-improvement. STRIVE begins with a warm-up phase, where the base model is fine-tuned on a small number of annotated examples to learn the structured reasoning design. It is then applied to generate reasoning chains for all training examples, selecting only those that are correct and structurally sound for subsequent self-improvement training. We demonstrate that STRIVE achieves significant improvements over baseline models, with a 31.4% performance gain over the base model and 20.7% over Chain of Thought on the HOVER datasets, highlighting its effectiveness.

事实验证自改进结构化推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。