让大模型用连续概念推理，提升逻辑能力
Improving Latent Reasoning in LLMs via Soft Concept Mixing
- 训练时引入软概念向量，融合到隐藏状态中
- 在五个推理基准上性能提升，训练过程稳定
- 适合关注模型内在推理机制的研究者
与人类在抽象概念空间中的推理不同，大语言模型（LLMs）通常通过生成离散标记进行推理，这可能限制其表达能力。近期研究《Soft Thinking》表明，通过软概念进行隐式推理是可行方向，但现有模型训练基于离散标记。为缩小推理中的软概念与训练中的离散标记之间的差距，本文提出软概念混合（SCM），一种感知软概念的训练方案，直接在训练过程中引入软表示。具体而言，SCM通过嵌入的加权平均构建软概念向量，并将其混入模型隐藏状态，以体现丰富的上下文信息。整个隐式推理过程通过强化学习（RL）进行优化。在五个推理基准上的实验表明，SCM提升了LLMs的推理性能，同时保持了稳定的训练动态。
原文摘要 · Abstract (English)
Unlike human reasoning in abstract conceptual spaces, large language models (LLMs) typically reason by generating discrete tokens, which potentially limit their expressive power. The recent work Soft Thinking has shown that LLMs' latent reasoning via soft concepts is a promising direction, but LLMs are trained on discrete tokens. To reduce this gap between the soft concepts in reasoning and the discrete tokens in training, we propose Soft Concept Mixing (SCM), a soft concept aware training scheme that directly exposes the model to soft representations during training. Specifically, SCM constructs a soft concept vector by forming a probability-weighted average of embeddings. Then, this vector is mixed into the model's hidden states, which embody rich contextual information. Finally, the entire latent reasoning process is optimized with Reinforcement Learning (RL). Experiments on five reasoning benchmarks demonstrate that SCM improves the reasoning performance of LLMs, and simultaneously maintains a stable training dynamic.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。