让大模型生成时保留概率分布信息，提升推理与文本质量
Text Generation Beyond Discrete Token Sampling
- 用贝叶斯方法融合采样结果与原始分布，生成连续输入
- 在数学推理等任务上，多个模型性能均有提升
- 无需训练，计算开销极小，适合实际部署
标准自回归生成中，大模型预测下一个词的概率分布，采样一个离散词后丢弃该分布，仅传递采样结果作为新输入。为保留分布中的丰富信息，我们提出无需训练的混合输入（Mixture of Inputs, MoI）方法。在标准生成流程后，将生成的离散词与先前丢弃的概率分布进行融合：利用贝叶斯估计，将分布视为先验，采样词作为观测，以连续后验期望替代传统 one-hot 向量作为新输入。该方法使模型在整个生成过程中维持更丰富的内部表示，显著提升文本质量与推理能力。在数学推理、代码生成和博士级问答任务中，对 QwQ-32B、Nemotron-Super-49B、Gemma-3-27B 与 DAPO-Qwen-32B 等多模型均实现持续性能提升，且无需额外训练，计算开销可忽略不计。
原文摘要 · Abstract (English)
In standard autoregressive generation, an LLM predicts the next-token distribution, samples a discrete token, and then discards the distribution, passing only the sampled token as new input. To preserve this distribution's rich information, we propose Mixture of Inputs (MoI), a training-free method for autoregressive generation. After generating a token following the standard paradigm, we construct a new input that blends the generated discrete token with the previously discarded token distribution. Specifically, we employ a Bayesian estimation method that treats the token distribution as the prior, the sampled token as the observation, and replaces the conventional one-hot vector with the continuous posterior expectation as the new model input. MoI allows the model to maintain a richer internal representation throughout the generation process, resulting in improved text quality and reasoning capabilities. On mathematical reasoning, code generation, and PhD-level QA tasks, MoI consistently improves performance across multiple models including QwQ-32B, Nemotron-Super-49B, Gemma-3-27B, and DAPO-Qwen-32B, with no additional training and negligible computational overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。