让多个推理路径互相交流，提升大模型的解题准确率。
LACE: Lattice Attention for Cross-thread Exploration
- 用交叉线程注意力机制让多条推理路径实时共享思路。
- 在多个基准上推理准确率提升超7个百分点。
- 适合需要多角度推理的复杂任务场景。
当前的大语言模型各自独立推理。尽管常并行采样多条推理路径，但这些路径互不交互，往往以相同方式失败。我们提出LACE框架，将孤立的推理尝试转变为协同并行过程。通过重构模型架构以支持跨线程注意力，LACE使并发推理路径能在推断过程中共享中间见解并相互修正错误。核心挑战在于缺乏自然的协作式训练数据，我们通过合成数据管道显式教导模型在不同路径间沟通与纠错。实验表明，这种统一探索显著优于标准并行搜索，在多个任务上推理准确率提升超过7个百分点。结果表明，允许并行推理路径交互可显著增强大语言模型的表现。
原文摘要 · Abstract (English)
Current large language models reason in isolation. Although it is common to sample multiple reasoning paths in parallel, these trajectories do not interact, and often fail in the same redundant ways. We introduce LACE, a framework that transforms reasoning from a collection of independent trials into a coordinated, parallel process. By repurposing the model architecture to enable cross-thread attention, LACE allows concurrent reasoning paths to share intermediate insights and correct one another during inference. A central challenge is the absence of natural training data that exhibits such collaborative behavior. We address this gap with a synthetic data pipeline that explicitly teaches models to communicate and error-correct across threads. Experiments show that this unified exploration substantially outperforms standard parallel search, improving reasoning accuracy by over 7 points. Our results suggest that large language models can be more effective when parallel reasoning paths are allowed to interact.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。