提出流水线解码器，实现上下文感知文本生成的并行加速。
Pipelined Decoder for Efficient Context-Aware Text Generation
- 将文本生成拆分为多个子序列并行处理，每步同时生成多个片段。
- 在问答、摘要等任务上速度提升显著，质量损失小于1%。
- 无需额外内存，适合对实时性要求高的生成场景。
自回归模型依赖先前所有已生成标记逐个生成新标记，虽保证高质量但严重限制生成速度。本文提出一种新型解码器架构——流水线解码器，可实现上下文感知文本生成的高效并行化。该方法同时启动多个子序列的生成，在每个时间步为每个子序列生成一个新标记，从而实现并行处理。在多项文本生成任务（包括问答、文本摘要和关键词生成）上的实验表明，该方法显著提升生成速度，且生成质量下降不足1%，无额外内存开销。
原文摘要 · Abstract (English)
As the basis of generative AI, an autoregressive model requires the generation of a new token depending on all the previously generated tokens, which brings high quality but also restricts the model to generate tokens one by one, forming a bottleneck limiting the generation speed. In this paper, we propose a new decoder architecture that efficiently generates text in parallel for context-aware generation tasks. Our proposed pipelined decoder initiates the generation of multiple subsequences simultaneously, and, at each time-step, it generates a new token for each subsequence to realize parallelism. Experiments on multiple text generation tasks, including question answering, text summarization, and keyphrase generation, show that our pipelined decoder significantly improves the generation speed without a significant loss of generation quality or additional memory consumption.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。