arXiv:2605.11936cs.AI2026-05被引 1

不用训练,随机向量注入也能提升大模型推理能力。

From Noise to Diversity: Random Embedding Injection in LLM Reasoning

论文配图:From Noise to Diversity: Random Embedding Injection in LLM Reasoning
图 1 · 摘自论文原文
  • 用随机向量替代训练过的提示,直接插入输入序列。
  • 在数学推理任务上,效果接近优化过的软提示。
  • 适合希望提升推理多样性的研究者和工程师。

近期的软提示研究通过在大模型输入中插入训练好的向量来改善推理,但其收益究竟来自学习内容还是注入行为本身尚未明确区分。本文研究随机软提示(RSP),完全省去训练步骤,仅将一组从预训练嵌入表均值与方差拟合的各向同性高斯分布中随机采样的嵌入向量序列附加到输入中。这些向量无任何学习内容,但在多个设置下的数学推理基准测试中表现可媲美优化后的软提示。其机制分为两个阶段:由于注意力需处理从未见过的随机位置,前几轮生成的词元分布趋于平坦,推理路径开始分叉;随着生成推进,该影响自然衰减,最终输出收敛至单一完成结果。我们发现,在推理过程中,RSP提升了早期词元多样性,结合温度采样后,显著扩大了Pass@N(N次尝试中至少一次正确的概率)。此外,该效应还可扩展至DAPO训练并带来实际性能提升。主要贡献包括:(i) RSP揭示了最简形式的软提示——免训练、每次重采样,为各类变体共有的结构效应提供统一视角;(ii) 理论与实证验证了其内在机制;(iii) 实现从推理到训练的延伸。

原文摘要 · Abstract (English)

Recent soft prompt research has tried to improve reasoning by inserting trained vectors into LLM inputs, yet whether the gain comes from the learned content or from the act of injection itself has not been carefully separated. We study Random Soft Prompts (RSPs), which drop the training step entirely and append a freshly drawn sequence of random embedding vectors to the input. Each RSP vector is sampled from an isotropic Gaussian fitted to the entrywise mean and variance of the pretrained embedding table; the sequence carries no learned content, and yet reaches accuracy comparable to optimized soft prompts on math reasoning benchmarks in several settings. The mechanism unfolds in two stages: because attention has to absorb a never-seen-before random position, the distribution over the first few generated tokens flattens and reasoning trajectories branch, and as generation continues this influence dilutes naturally so the response commits to a single completion. We show that during inference RSPs lift early-stage token diversity and, combined with temperature sampling, widen Pass@N, the probability that at least one out of N attempts is correct. Beyond inference, we carry the same effect into DAPO training and demonstrate practical gains. Our contributions are: (i) RSP isolates the simplest form of soft prompt -- training-free, freshly resampled -- providing a unified lens for the structural effect of injection that variants otherwise differing in training and form all share; (ii) a theoretical and empirical validation of the underlying mechanism; and (iii) an extension from inference to training.

大模型推理软提示随机注入多样性增强

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。