解决大模型长程推理中分解过度导致的不可恢复错误问题
LEAD: Breaking the No-Recovery Bottleneck in Long-Horizon Reasoning
- 引入前瞻验证与重叠回溯，动态调整推理分解粒度
- 使o4-mini模型在13步跳棋任务中稳定求解，突破前人极限
- 适合需要高可靠性长链推理的应用场景
大型语言模型在长时序执行中仍不稳定，即使提供高层策略。在控制性算法谜题上评估发现，虽然分解对稳定性至关重要，但过度分解会引发'不可恢复瓶颈'。该瓶颈源于错误分布高度不均，少数'困难'步骤的持续错误无法纠正。为此，我们提出前瞻增强原子分解（LEAD）：通过短时展望验证和重叠轨迹聚合，在保持稳定性的同时保留局部纠错能力。该方法使o4-mini模型成功求解复杂度n=13的跳棋问题，而极端分解在n=11后即失效。
原文摘要 · Abstract (English)
Long-horizon execution in Large Language Models (LLMs) remains unstable even when high-level strategies are provided. Evaluating on controlled algorithmic puzzles, we demonstrate that while decomposition is essential for stability, extreme decomposition creates a "no-recovery bottleneck". We show that this bottleneck becomes critical due to highly non-uniform error distribution, where consistent errors on a few "hard" steps become irreversible. To address this, we propose Lookahead-Enhanced Atomic Decomposition (LEAD). By incorporating short-horizon future validation and aggregating overlapping rollouts, LEAD provides enough isolation to maintain stability while retaining enough local context to correct errors. This enables the o4-mini model to solve Checkers Jumping up to complexity $n=13$, whereas extreme decomposition fails beyond $n=11$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。