用时间自蒸馏让大模型一次生成32个字，速度超自回归模型8倍
Beyond Autoregression: Fast LLMs via Self-Distillation Through Time
- 通过时间自蒸馏技术，让离散扩散模型一次性生成32个词元
- 1.3B参数模型无需缓存时，推理速度比带缓存的自回归模型快8倍
- 适合追求高速生成且对文本质量有要求的场景
自回归大语言模型在众多任务中表现优异，但其逐词生成机制导致延迟明显。近期研究表明，推理时通过搜索与重复采样可提升定理证明、代码生成和对齐等任务的表现。本文展示，扩散语言模型可一次性生成至少32个词元，且在文本质量和LAMBADA理解基准上优于自回归模型。这得益于一种新型离散扩散模型的蒸馏方法，将推理步数减少32至64倍。在1.3B参数规模下，即使不使用缓存，扩散模型生成速度也可达自回归模型（启用KV缓存）的8倍，未来加入缓存有望进一步提升。该方法在最大860M参数的扩散模型上同样有效。
原文摘要 · Abstract (English)
Autoregressive (AR) Large Language Models (LLMs) have demonstrated significant success across numerous tasks. However, the AR modeling paradigm presents certain limitations; for instance, contemporary autoregressive LLMs are trained to generate one token at a time, which can result in noticeable latency. Recent advances have indicated that search and repeated sampling can enhance performance in various applications, such as theorem proving, code generation, and alignment, by utilizing greater computational resources during inference. In this study, we demonstrate that diffusion language models are capable of generating at least 32 tokens simultaneously, while exceeding the performance of AR models in text quality and on the LAMBADA natural language understanding benchmark. This outcome is achieved through a novel distillation method for discrete diffusion models, which reduces the number of inference steps by a factor of 32-64. Practically, at the 1.3B parameters scale, diffusion models, even without caching, can generate tokens at a rate that is up to 8 times faster than AR models employing KV-caching, and we anticipate further improvements with the inclusion of caching. Moreover, we demonstrate the efficacy of our approach for diffusion language models with up to 860M parameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。