通过双向压缩与生成提升大模型推理速度,无需验证即可高效出结果。
Pair-In, Pair-Out: Latent Multi-Token Prediction for Efficient LLMs

- 将输入压缩与输出预测视为对称操作,统一优化前后处理
- 在AIME2025等数据集上提升准确率最高达+7.15点,速度提升2.64倍
- 轻量信心头配合策略蒸馏,无需额外验证开销,适合部署场景
长链思维推理使自回归解码成为现代大语言模型的主要推理开销。现有方法或聚焦输入侧(潜在压缩),或关注输出侧(推测解码与多标记预测,MTP),但二者独立发展。且输出侧方法需昂贵的验证步骤来确认不可靠的草稿标记。为此,我们提出「成对输入、成对输出(PIPO)」,将潜在压缩器与MTP头视为镜像操作:压缩器将两个输入标记合并为一个隐状态,而MTP头将一个隐藏状态展开为一个额外输出标记。为在不牺牲可靠性的情况下消除验证成本,PIPO训练了一个轻量级置信度头,决定是否接受草稿标记。我们发现,在策略蒸馏(OPD)中自然匹配推测解码的拒绝采样准则,因此可与之联合训练,几乎无额外开销。在使用Qwen3.5-4B和9B骨干模型的AIME 2025、GPQA-Diamond、LiveCodeBench v6和LongBench v2上实验表明,相比常规解码,PIPO最高提升pass@4达+7.15点,同时实现最高2.64倍首标记延迟加速和2.07倍每标记延迟加速。
原文摘要 · Abstract (English)
Long chain-of-thought reasoning has made autoregressive decoding the dominant inference cost of modern large language models. Existing methods target either the input side (latent compression) or the output side (speculative decoding and multi-token prediction, MTP), but the two lines of work have been pursued independently. Moreover, output-side methods must incur an expensive verifier pass to validate the unreliable draft tokens predicted by MTP. To address these issues, we propose \textbf{Pair-In, Pair-Out (PIPO)}, which unifies both sides by viewing a latent compressor and an MTP head as mirror-image operations: the compressor folds two input tokens into one latent representation, while the MTP head unfolds one hidden state into one additional output token. To remove the verifier cost without sacrificing reliability, PIPO trains a lightweight confidence head that decides whether draft tokens should be accepted. We observe that On-Policy Distillation (OPD) naturally matches the rejection-sampling criterion of speculative decoding, so the confidence head can be trained alongside OPD with negligible extra cost. Experiments on AIME 2025, GPQA-Diamond, LiveCodeBench v6, and LongBench v2 with Qwen3.5-4B and 9B backbones show that PIPO improves pass@4 over regular decoding by up to $+7.15$ points, while delivering up to $2.64\times$ first-token-latency and $2.07\times$ per-token-latency speedups. Project Page: GitHub.com/RedAI-Infra/PIPO.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。