让大模型自己学会并行生成，提升推理速度而不明显牺牲质量。
Learning to Keep a Promise: Scaling Language Model Decoding Parallelism with Learned Asynchronous Decoding
- 用自学习语言标注并行生成机会，替代人工规则。
- 在指令遵循任务上实现1.21到1.93倍的加速，质量波动在±7.1%内。
- 适合追求推理效率又不希望大幅降质的部署场景。
传统自回归大模型解码逐个生成词元，效率受限。现有并行解码方法依赖手工规则识别语义独立段落，灵活性差。本文提出PASTA系统，通过训练大模型使用自定义标注语言（PASTA-LANG）表达自身响应中的语义独立性，由解释器实时调度并行生成。经两阶段微调，模型能同时优化响应质量与解码速度。在AlpacaEval基准测试中，本方法在解码速度与质量之间实现帕累托优势：几何平均加速1.21倍至1.93倍，对应长度控制下的胜率变化为+2.2%至-7.1%，优于序列基线。
原文摘要 · Abstract (English)
Decoding with autoregressive large language models (LLMs) traditionally occurs sequentially, generating one token after another. An emerging line of work explored parallel decoding by identifying and simultaneously generating semantically independent chunks of LLM responses. However, these techniques rely on hand-crafted heuristics tied to syntactic structures like lists and paragraphs, making them rigid and imprecise. We present PASTA, a learning-based system that teaches LLMs to identify semantic independence and express parallel decoding opportunities in their own responses. At its core are PASTA-LANG and its interpreter: PASTA-LANG is an annotation language that enables LLMs to express semantic independence in their own responses; the language interpreter acts on these annotations to orchestrate parallel decoding on-the-fly at inference time. Through a two-stage finetuning process, we train LLMs to generate PASTA-LANG annotations that optimize both response quality and decoding speed. Evaluation on AlpacaEval, an instruction following benchmark, shows that our approach Pareto-dominates existing methods in terms of decoding speed and response quality; our results demonstrate geometric mean speedups ranging from 1.21x to 1.93x with corresponding quality changes of +2.2% to -7.1%, measured by length-controlled win rates against sequential decoding baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。