提出新型字节级语言模型，生成速度提升超50%。
Fast Byte Latent Transformer
- 引入块级扩散训练，实现多字节并行生成。
- 三种方法使生成效率提升，内存带宽成本降50%以上。
- 适合追求高速生成的字节级模型应用者。
近期的字节级语言模型在不依赖子词词汇的情况下达到与词元级模型相当的性能，但其实际应用受限于逐字节自回归生成的缓慢速度。本文通过新的训练与生成技术解决这一瓶颈，提出字节潜在变换器（BLT）。首先引入BLT扩散（BLT-D），一种新模型及最快变体，通过辅助的块级扩散目标与标准的下一个字节预测损失联合训练，使推理阶段每步可并行生成多个字节，显著减少生成序列所需的前向传播次数。其次提出两种受推测解码启发的改进：BLT自推测（BLT-S），在局部解码器越过正常块边界继续生成候选字节，再通过一次全模型前向传播验证；以及BLT扩散+验证（BLT-DV），在扩散生成后增加自回归验证步骤。所有方法在生成任务中估计的内存带宽成本均比传统BLT降低超过50%。各类方法各具优势，共同消除了字节级语言模型实用化的关键障碍。
原文摘要 · Abstract (English)
Recent byte-level language models (LMs) match the performance of token-level models without relying on subword vocabularies, yet their utility is limited by slow, byte-by-byte autoregressive generation. We address this bottleneck in the Byte Latent Transformer (BLT) through new training and generation techniques. First, we introduce BLT Diffusion (BLT-D), a new model and our fastest BLT variant, trained with an auxiliary block-wise diffusion objective alongside the standard next-byte prediction loss. This enables an inference procedure that generates multiple bytes in parallel per decoding step, substantially reducing the number of forward passes required to generate a sequence. Second, we propose two extensions inspired by speculative decoding that trade some of this speed for higher generation quality: BLT Self-speculation (BLT-S), in which BLT's local decoder continues generating past its normal patch boundaries to draft bytes, which are then verified with a single full-model forward pass; and BLT Diffusion+Verification (BLT-DV), which augments BLT-D with an autoregressive verification step after diffusion-based generation. All methods may achieve an estimated memory-bandwidth cost over 50% lower than BLT on generation tasks. Each approach offers its own unique advantages, together removing key barriers to the practical use of byte-level LMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。