让大模型生成更快更省内存，同时保持精度。
Orthrus: Memory-Efficient Parallel Token Generation via Dual-View Diffusion
- 用双视角架构并行生成文本，保留原有模型精度。
- 相比自回归方法快7.8倍，内存开销仅增加常数级别。
- 适合需要高速推理又不牺牲质量的场景。
我们提出 Orthrus，一种简单高效的双架构框架，将自回归大语言模型（LLM）的精确生成能力与扩散模型的高速并行生成能力相结合。标准自回归解码的串行特性是高吞吐推理的根本瓶颈。尽管扩散语言模型尝试通过并行生成突破这一限制，但仍面临性能下降、训练成本高及缺乏严格收敛保证等问题。Orthrus 从本质上解决了这一矛盾。该框架可无缝集成到现有 Transformer 模型中，通过在冻结的 LLM 上添加一个轻量可训练模块，构建出与标准自回归视图并行的扩散视图。两者共享相同的高保真键值（KV）缓存：自回归头负责上下文预填充以构建准确的 KV 表示，扩散头则执行并行生成。通过两视图间的精确一致性机制，Orthrus 实现无损推理，在仅增加 O(1) 内存缓存开销和极少参数的前提下，实现高达 7.8 倍的加速。
原文摘要 · Abstract (English)
We introduce Orthrus, a simple and efficient dual-architecture framework that unifies the exact generation fidelity of autoregressive Large Language Models (LLMs) with the high-speed parallel token generation of diffusion models. The sequential nature of standard autoregressive decoding represents a fundamental bottleneck for high-throughput inference. While diffusion language models attempt to break this barrier via parallel generation, they suffer from significant performance degradation, high training costs, and a lack of rigorous convergence guarantees. Orthrus resolves this dichotomy natively. Designed to seamlessly integrate into existing Transformers, the framework augments a frozen LLM with a lightweight, trainable module to create a parallel diffusion view alongside the standard autoregressive view. In this unified system, both views attend to the exact same high-fidelity Key-Value (KV) cache; the autoregressive head executes context pre-filling to construct accurate KV representations, while the diffusion head executes parallel generation. By employing an exact consensus mechanism between the two views, Orthrus guarantees lossless inference, delivering up to a 7.8x speedup with only an O(1) memory cache overhead and minimal parameter additions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。