基于离散扩散的快速代码生成模型，推理速度达2146词元/秒。
Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference
- 采用离散扩散机制实现并行生成，摆脱逐词解码延迟。
- 在H20 GPU上实现2146词元/秒的推理速度，优于现有模型。
- 适合追求高速生成的代码生成场景，尤其适合实时应用。
我们提出Seed Diffusion Preview，一种基于离散状态扩散的大规模语言模型，具备显著的推理速度优势。得益于非序列化的并行生成方式，离散扩散模型有效缓解了传统逐词解码带来的固有延迟，如近期提出的Mercury Coder和Gemini Diffusion所示。Seed Diffusion Preview在H20 GPU上实现了2,146词元/秒的推理速度，同时在一系列标准代码评估基准上保持了具有竞争力的性能，显著超越当前的Mercury和Gemini Diffusion，在代码模型的速度-质量帕累托前沿上建立了新的基准。
原文摘要 · Abstract (English)
We present Seed Diffusion Preview, a large-scale language model based on discrete-state diffusion, offering remarkably fast inference speed. Thanks to non-sequential, parallel generation, discrete diffusion models provide a notable speedup to mitigate the inherent latency of token-by-token decoding, as demonstrated recently (e.g., Mercury Coder, Gemini Diffusion). Seed Diffusion Preview achieves an inference speed of 2,146 token/s over H20 GPUs while maintaining competitive performance across a sweep of standard code evaluation benchmarks, significantly faster than contemporary Mercury and Gemini Diffusion, establishing new state of the art on the speed-quality Pareto frontier for code models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。