让大模型并行生成的多个回答互相依赖，提升质量与一致性。
Generalized Parallel Scaling with Interdependent Generations
- 将批量隐藏状态视为整体张量，实现并行生成间的信息互通。
- 仅增加2.8%-5.1%参数，使强化学习奖励验证下的准确率提升最高达39%。
- 一次训练即可适配任意生成宽度，兼容各种后处理聚合方法。
并行大模型推理需对单个输入生成多个响应（N>1），但传统方法使这些响应相互独立，浪费计算资源且无法共享信息。这与长度扩展中利用历史计算形成对比。为此，本文提出Bridge方法，将批量隐藏状态视为整体张量而非独立片段，实现并行响应间的依赖生成。仅新增2.8%-5.1%参数，便使基于可验证奖励的强化学习相对平均准确率提升最高达39%，并显著提高正确响应的一致性。Bridge仅需一次训练，即可适配任意生成宽度，性能全面优于独立生成，开启一种更通用的并行扩展模式，有效利用序列间信息，兼容任意后生成聚合技术。
原文摘要 · Abstract (English)
Parallel LLM inference scaling involves sampling a set of $N>1$ responses for a single input prompt. However, these $N$ parallel responses tend to be generated independently from each other, partitioning compute resources and leaving potentially useful information in one generation untapped by others. This is in contrast to response length scaling where past computation is used in all future steps. For higher quality responses and response sets, we propose Bridge to generate interdependent responses in parallel by rethinking batched LLM hidden states as holistic tensors rather than independent slices. With only a small amount (2.8%-5.1%) of new parameters, Bridge improves the relative mean accuracy gains from reinforcement learning with verifiable rewards by up to 39% and boosts consistency of correct responses. Trained once, Bridge scales to any generation width, all with greater performance than independent generations, unlocking a more general mode of parallel scaling that effectively leverages information between sequences, compatible with any post-generation aggregation technique.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。