用轻量辅助模型生成连续思维令牌，提升大模型推理效率
SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMs
- 用固定小模型生成软思维令牌，再映射到大模型空间
- 在5个基准上显著提升推理性能，仅需少量参数微调
- 无需修改大模型，适合已有良好零样本表现的模型
链式思维（CoT）推理使大语言模型通过生成中间推理步骤来解决复杂任务。然而，现有方法多依赖离散词元解码，受限于词汇空间且未必最优。尽管有研究探索连续空间推理，但通常需要全模型微调，易引发灾难性遗忘，难以应用于已在零样本设置下表现良好的先进大模型。为此，我们提出一种无需修改大模型的新型连续空间推理方法：使用轻量级固定助手模型为具体实例生成特定的软思维词元，再通过可训练投影模块将其映射至大模型表示空间。在五个推理基准上的实验表明，该方法通过监督式、参数高效的微调显著提升了大模型的推理性能。源代码已公开于 https://github.com/xuyige/SoftCoT。
原文摘要 · Abstract (English)
Chain-of-Thought (CoT) reasoning enables Large Language Models (LLMs) to solve complex reasoning tasks by generating intermediate reasoning steps. However, most existing approaches focus on hard token decoding, which constrains reasoning within the discrete vocabulary space and may not always be optimal. While recent efforts explore continuous-space reasoning, they often require full-model fine-tuning and suffer from catastrophic forgetting, limiting their applicability to state-of-the-art LLMs that already perform well in zero-shot settings with a proper instruction. To address this challenge, we propose a novel approach for continuous-space reasoning that does not require modifying the LLM. Specifically, we employ a lightweight fixed assistant model to speculatively generate instance-specific soft thought tokens as the initial chain of thoughts, which are then mapped into the LLM's representation space via a trainable projection module. Experimental results on five reasoning benchmarks demonstrate that our method enhances LLM reasoning performance through supervised, parameter-efficient fine-tuning. Source code is available at https://github.com/xuyige/SoftCoT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。