arXiv:2502.02390cs.CLcs.AI2025-02EMNLP被引 16

通过联想记忆与搜索结合,让大模型像人一样逐步思考。

CoAT: Chain-of-Associated-Thoughts Framework for Enhancing Large Language Models Reasoning

  • 用蒙特卡洛树搜索+动态联想记忆,拓展推理路径
  • 在多个数据集上提升10%以上,自研数据集增益超15%
  • 适合需要深度推理的复杂任务,如多跳问答

大语言模型普遍采用‘快速思考’模式,仅基于单次输入生成结果。随着OpenAI-o1等‘慢思考’技术兴起,研究关注人类思维过程的模拟。受此启发,我们提出链式联想思维(CoAT)框架,将蒙特卡洛树搜索(MCTS)与动态联想记忆机制结合,实现知识的持续补充与推理路径的结构化探索。该框架可实时更新知识库,回溯并修正先前推理,确保输出准确且全面。在多种生成与推理任务中验证,CoAT在开源多跳推理数据集(HotpotQA、MuSiQue)上性能提升超10%,在自研CRB数据集上提升超过15%。

原文摘要 · Abstract (English)

Research on LLM technologies is rapidly emerging, with most of them employ a 'fast thinking' approach to inference. Most LLMs generate the final result based solely on a single query and LLM's reasoning capabilities. However, with the advent of OpenAI-o1, 'slow thinking' techniques have garnered increasing attention because its process is closer to the human thought process. Inspired by the human ability to constantly associate and replenish knowledge during thinking, we developed the novel Chain-of-Associated-Thoughts (CoAT) framework, which introduces an innovative synergy between the Monte Carlo Tree Search (MCTS) algorithm and a dynamic mechanism for integrating new key information, termed 'associative memory'. By combining the structured exploration capabilities of MCTS with the adaptive learning capacity of associative memory, CoAT significantly expands the LLM search space, enabling our framework to explore diverse reasoning pathways and dynamically update its knowledge base in real-time. This allows the framework to not only revisit and refine earlier inferences but also adaptively incorporate evolving information, ensuring that the final output is both accurate and comprehensive. We validate CoAT's effectiveness across a variety of generative and reasoning tasks. Quantitative experiments show that CoAT achieves over 10% performance improvement on open-source multi-hop reasoning datasets (HotpotQA, MuSiQue) and more than 15% gain on our proprietary CRB dataset.

大模型推理链式思维搜索增强多跳问答

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。