让大模型具备双向逻辑修复能力,从错误中精准修补推理链。
Imbuing Large Language Models with Bidirectional Logic for Robust Chain Repair

- 将错误推理段落重构为填空任务,用前后验证信息引导修复。
- 在三个基准上达到最优性能,平均减少31.2%的推理耗 token。
- 适合需要高可靠性推理的数学、代码生成等场景。
大型语言模型的自回归思维链推理本质上是单向的:每一步仅依赖前面的标记。这种单向归纳偏置使模型极易出现错误雪崩——早期一个逻辑或算术错误会不可逆地污染整个推理链。本文提出目标导向推理补全( extit{TRI}),赋予解码器仅架构的Transformer原生的“目标条件桥接”能力。核心思想是将错误推理段视为填空任务:给定已验证的前缀前提 $P$、下游里程碑 $S$ 以及原始问题 $Q$,模型需严格完整地合成连接 $P$ 到 $S$ 的逻辑桥梁 $M$。为适配标准因果架构,引入前缀-后缀-中间(PSM)序列重排与三个非重叠哨兵标记,使 $M$ 能同时关注 $P$ 与 $S$,无需修改自注意力机制。训练分两阶段:(i) 在形式数学语料中提取符号验证的 $(P, S, M)$ 三元组进行监督微调;(ii) 使用确定性符号验证器(Lean 4 / Python)作为唯一奖励信号,通过直接偏好优化(DPO)消除大模型裁判的谄媚倾向。推理时,TRI 作为双系统循环中的手术式修复模块:因果草稿模型生成初始推理路径,验证器定位失败点,TRI 仅补全受损段落,保留已验证部分。全面实验表明,TRI 在三个基准上均达当前最佳表现,且每题推理耗能降低 31.2%。
原文摘要 · Abstract (English)
Autoregressive chain-of-thought (CoT) reasoning in large language models (LLMs) is fundamentally forward-directed: each step conditions only on prior tokens. This unidirectional inductive bias renders even capable models susceptible to error snowballing, wherein a single logical or arithmetic mistake in an early step irreversibly corrupts the entire reasoning chain. We introduce Teleological Reasoning Infilling (\TRI{}), a training framework that endows decoder-only transformers with a native \emph{goal-conditioned bridging} capability. The key insight is to reframe erroneous reasoning segments as fill-in-the-middle (FIM) tasks: given a verified prefix premise $P$, a verified downstream milestone $S$, and the original query $Q$, the model must synthesise the logical bridge $M$ that connects $P$ to $S$ rigorously and completely. To achieve this with standard causal architectures, we introduce a Prefix-Suffix-Middle (PSM) sequence rearrangement with three non-overlapping sentinel tokens, enabling $M$ to attend to both $P$ and $S$ without any structural modification to the self-attention mechanism. Training proceeds in two stages: (i) Supervised Fine-Tuning (SFT) on symbolically verified $(P, S, M)$ triples extracted from formal mathematics corpora, and (ii) Direct Preference Optimisation (DPO) with a deterministic symbolic verifier (Lean 4 / Python) as the sole reward oracle, eliminating LLM-judge sycophancy. At inference, TRI operates as a surgical repair module within a dual-system loop: a causal draft model generates an initial trace, the verifier pinpoints failures, and TRI infills only the damaged segment, leaving verified sections intact. Comprehensive experiments on three benchmarks demonstrate that TRI achieves state-of-the-art performance across all tasks, while reducing per-problem token expenditure by 31.2%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。