让大模型学会像人一样思考：教它反思自己的推理过程
Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought
- 通过元思维链（Meta-CoT）显式建模推理背后的思考逻辑
- 在多个任务上实现比传统CoT更优的推理准确率，提升显著
- 适合研究可解释推理、智能体决策和高级AI系统构建的人
我们提出一种新框架——元思维链（Meta Chain-of-Thought, Meta-CoT），通过显式建模生成特定思维链所需的底层推理过程，扩展了传统的思维链方法。基于前沿大模型的实证研究发现其表现出类似上下文搜索的行为。本文探索了通过过程监督、合成数据生成和搜索算法生成Meta-CoT的方法，并提出了一个具体的训练流水线：结合指令微调与线性化搜索轨迹，再通过后训练强化学习优化。最后讨论了可扩展性规律、验证器角色以及发现新型推理算法的潜力。该工作为实现大模型中的元思维链提供了理论与实践路径，推动人工智能向更强大、类人的推理能力演进。
原文摘要 · Abstract (English)
We propose a novel framework, Meta Chain-of-Thought (Meta-CoT), which extends traditional Chain-of-Thought (CoT) by explicitly modeling the underlying reasoning required to arrive at a particular CoT. We present empirical evidence from state-of-the-art models exhibiting behaviors consistent with in-context search, and explore methods for producing Meta-CoT via process supervision, synthetic data generation, and search algorithms. Finally, we outline a concrete pipeline for training a model to produce Meta-CoTs, incorporating instruction tuning with linearized search traces and reinforcement learning post-training. Finally, we discuss open research questions, including scaling laws, verifier roles, and the potential for discovering novel reasoning algorithms. This work provides a theoretical and practical roadmap to enable Meta-CoT in LLMs, paving the way for more powerful and human-like reasoning in artificial intelligence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。