用因果增强与双向搜索提升大模型长程推理能力
CreDes: Causal Reasoning Enhancement and Dual-End Searching for Solving Long-Range Reasoning Problems using LLMs
- 引入因果关系增强机制，确保推理与状态转移一致
- 双向搜索策略在因果概率树上同时从起点和目标出发
- 相比单步链式推理，准确率和效率显著提升
大语言模型在处理涉及长程推理的组合优化问题时存在局限，部分源于因果幻觉和庞大的搜索空间。针对因果幻觉（即推理与状态转移不一致），本文提出因果关系增强（CRE）机制，结合因果干预与个体处理效应（ITE），确保每一步推理与状态转移之间的稳固因果关系。针对长因果链条和巨大搜索空间导致的性能瓶颈，提出双端搜索（DES）方法，通过在因果概率树上同时从初始状态和目标状态出发寻找解。将CRE与DES融合形成CreDes模型，实现多步并行推理，避免传统链式思维（CoT）中逐次推理的低效问题。实验表明，CreDes在长程推理任务中显著优于现有最先进方法，在准确率和时间效率方面均有明显提升。
原文摘要 · Abstract (English)
Large language models (LLMs) have demonstrated limitations in handling combinatorial optimization problems involving long-range reasoning, partially due to causal hallucinations and huge search space. As for causal hallucinations, i.e., the inconsistency between reasoning and corresponding state transition, this paper introduces the Causal Relationship Enhancement (CRE) mechanism combining cause-effect interventions and the Individual Treatment Effect (ITE) to guarantee the solid causal rightness between each step of reasoning and state transition. As for the long causal range and huge search space limiting the performances of existing models featuring single-direction search, a Dual-End Searching (DES) approach is proposed to seek solutions by simultaneously starting from both the initial and goal states on the causal probability tree. By integrating CRE and DES (CreDes), our model has realized simultaneous multi-step reasoning, circumventing the inefficiencies from cascading multiple one-step reasoning like the Chain-of-Thought (CoT). Experiments demonstrate that CreDes significantly outperforms existing State-Of-The-Art (SOTA) solutions in long-range reasoning tasks in terms of both accuracy and time efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。