将思维链压缩到连续空间，让大模型更高效地推理
CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation
- 用自蒸馏方法把自然语言思维链转成连续向量表示
- 在GSM8k上达到与显式思维链相当的准确率，压缩率达3.1倍
- 适合追求推理效率与模型轻量化的研究者和开发者
思维链（CoT）通过自然语言逐步推理提升了大语言模型的能力。但使用潜在连续空间进行推理可能带来更高的效率和鲁棒性。以往隐式思维链方法试图完全避开自然语言，在连续空间中推理，但表现始终逊于标准显式思维链。我们提出CODI（通过自蒸馏实现连续思维链），一种新型训练框架，能有效将自然语言思维链压缩至连续空间。CODI联合训练教师任务（显式思维链）与学生任务（隐式思维链），通过对指定标记的隐藏状态对齐，实现推理能力从语言空间到连续空间的迁移。实验表明，CODI是首个在GPT-2规模下达到显式思维链性能的隐式方法，在GSM8k上实现3.1倍压缩率，准确率比此前最优方法提升28.2%。该方法还展现出良好鲁棒性、可泛化至复杂数据集及可解释性。结果验证了大模型不仅能在自然语言中推理，也能在潜在连续空间中高效推理。代码已开源：https://github.com/zhenyi4/codi。
原文摘要 · Abstract (English)
Chain-of-Thought (CoT) reasoning enhances Large Language Models (LLMs) by encouraging step-by-step reasoning in natural language. However, leveraging a latent continuous space for reasoning may offer benefits in terms of both efficiency and robustness. Prior implicit CoT methods attempt to bypass language completely by reasoning in continuous space but have consistently underperformed compared to the standard explicit CoT approach. We introduce CODI (Continuous Chain-of-Thought via Self-Distillation), a novel training framework that effectively compresses natural language CoT into continuous space. CODI jointly trains a teacher task (Explicit CoT) and a student task (Implicit CoT), distilling the reasoning ability from language into continuous space by aligning the hidden states of a designated token. Our experiments show that CODI is the first implicit CoT approach to match the performance of explicit CoT on GSM8k at the GPT-2 scale, achieving a 3.1x compression rate and outperforming the previous state-of-the-art by 28.2% in accuracy. CODI also demonstrates robustness, generalizable to complex datasets, and interpretability. These results validate that LLMs can reason effectively not only in natural language, but also in a latent continuous space. Code is available at https://github.com/zhenyi4/codi.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。