通过构建真假选项的对比推理路径,提升模型逻辑阅读理解能力。
Thought-Path Contrastive Learning via Premise-Oriented Data Augmentation for Logical Reading Comprehension
- 基于前提生成真假选项的推理链条,增强分析全面性。
- 在ReClor和LogiQA 2.0上显著超越基线,提升幅度达数个百分点。
- 适合需要提升推理泛化能力的研究者与开发者。
逻辑阅读理解是一项挑战性任务,要求模型深入理解文本语义并进行推理以得出正确答案。以往研究多依赖思维链(CoT)或数据增强来提升推理能力,但现有方法仅分析正确选项,忽略错误选项;且基于规则的数据增强生成的上下文缺乏多样性和连贯性。为此,本文提出前提导向的数据增强(PODA)框架,能生成包含正确与错误选项分析的思维链,并从错误选项中构建多样化、高质量的反事实上下文。该框架将前提总结与识别融入推理过程,并通过多步提示构建反事实情境。为进一步强化模型区分不同推理路径的能力,引入一种新型思想路径对比学习方法,比较原始样本与反事实样本间的推理路径。在三个代表性大模型上的实验表明,该方法在两个高难度逻辑推理基准(ReClor 和 LogiQA 2.0)上显著优于基线。代码与数据已开源。
原文摘要 · Abstract (English)
Logical reading comprehension is a challenging task that entails grasping the underlying semantics of text and applying reasoning to deduce the correct answer. Prior researches have primarily focused on enhancing logical reasoning capabilities through Chain-of-Thought (CoT) or data augmentation. However, previous work constructing chain-of-thought rationales concentrates solely on analyzing correct options, neglecting the incorrect alternatives. Addtionally, earlier efforts on data augmentation by altering contexts rely on rule-based methods, which result in generated contexts that lack diversity and coherence. To address these issues, we propose a Premise-Oriented Data Augmentation (PODA) framework. This framework can generate CoT rationales including analyses for both correct and incorrect options, while constructing diverse and high-quality counterfactual contexts from incorrect candidate options. We integrate summarizing premises and identifying premises for each option into rationales. Subsequently, we employ multi-step prompts with identified premises to construct counterfactual context. To facilitate the model's capabilities to better differentiate the reasoning process associated with each option, we introduce a novel thought-path contrastive learning method that compares reasoning paths between the original and counterfactual samples. Experimental results on three representative LLMs demonstrate that our method can improve the baselines substantially across two challenging logical reasoning benchmarks (ReClor and LogiQA 2.0). The data and code are released at https://github.com/lalalamdbf/TPReasoner.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。