让大模型像人一样软性推理，用更短序列解决复杂数学题。
Multiplex Thinking: Reasoning via Token-wise Branch-and-Merge
- 每步采样多个候选词，融合成一个连续的混合标记。
- 在数学推理任务上，从单次到上千次通过率均优于现有方法。
- 自适应调节：自信时像传统链式推理，不确定时压缩多种可能。
大型语言模型在复杂推理任务中常依赖链式思维（CoT），但导致长而低效的词元序列。人类则倾向于以概率分布形式软性推理。受此启发，我们提出多重思维（Multiplex Thinking），一种随机软推理机制：每一步采样 K 个候选词，并将其嵌入向量聚合为单一连续的混合标记。该方法保留词汇嵌入先验与标准离散生成的采样动态，同时在混合路径上诱导可处理的概率分布，使策略可直接通过在线强化学习优化。关键优势在于自适应性：当模型自信时，混合标记趋近离散，表现如标准 CoT；当不确定时，紧凑表示多个合理后续步骤而不增加序列长度。在多个高难度数学推理基准上，多重思维从 Pass@1 到 Pass@1024 均持续超越强基线，且生成序列更短。代码与检查点已开源：https://github.com/GMLR-Penn/Multiplex-Thinking。
原文摘要 · Abstract (English)
Large language models often solve complex reasoning tasks more effectively with Chain-of-Thought (CoT), but at the cost of long, low-bandwidth token sequences. Humans, by contrast, often reason softly by maintaining a distribution over plausible next steps. Motivated by this, we propose Multiplex Thinking, a stochastic soft reasoning mechanism that, at each thinking step, samples K candidate tokens and aggregates their embeddings into a single continuous multiplex token. This preserves the vocabulary embedding prior and the sampling dynamics of standard discrete generation, while inducing a tractable probability distribution over multiplex rollouts. Consequently, multiplex trajectories can be directly optimized with on-policy reinforcement learning (RL). Importantly, Multiplex Thinking is self-adaptive: when the model is confident, the multiplex token is nearly discrete and behaves like standard CoT; when it is uncertain, it compactly represents multiple plausible next steps without increasing sequence length. Across challenging math reasoning benchmarks, Multiplex Thinking consistently outperforms strong discrete CoT and RL baselines from Pass@1 through Pass@1024, while producing shorter sequences. The code and checkpoints are available at https://github.com/GMLR-Penn/Multiplex-Thinking.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。