提出FutureFill，让卷积序列模型生成速度更快、内存更省。
FutureFill: Fast Generation from Convolutional Sequence Models
- 用新方法重写卷积序列模型的生成过程，提升效率
- 生成时间从二次方降到近线性，缓存大小仅随输出长度增长
- 适合需要快速生成长序列的场景，如语音或文本合成
我们针对序列预测模型中自回归生成效率低的问题，提出FutureFill——一种适用于任意基于卷积算子的序列预测算法的通用快速生成方法。该方法将生成时间从与上下文长度成平方关系降低至近线性关系。此外，在给定提示生成时，其预填充缓存大小仅随生成词元数量增长，通常远小于标准卷积或注意力模型所需的缓存。我们在合成任务上验证了理论结论，并在深度卷积序列预测模型上展示了显著的效率提升。
原文摘要 · Abstract (English)
We address the challenge of efficient auto-regressive generation in sequence prediction models by introducing FutureFill, a general-purpose fast generation method for any sequence prediction algorithm based on convolutional operators. FutureFill reduces generation time from quadratic to quasilinear in the context length. Moreover, when generating from a prompt, it requires a prefill cache whose size grows only with the number of tokens to be generated, often much smaller than the caches required by standard convolutional or attention based models. We validate our theoretical claims with experiments on synthetic tasks and demonstrate substantial efficiency gains when generating from a deep convolutional sequence prediction model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。