基于扩散模型的超快代码生成大模型，速度比现有优化模型快10倍。
Mercury: Ultra-Fast Language Models Based on Diffusion
- 采用扩散机制并行预测多个词元，通过Transformer架构实现
- 在H100上达1109和737 tokens/sec吞吐，速度领先10倍且质量相当
- 适合追求极致推理速度的开发者与工业级代码生成场景
我们提出Mercury，一种基于扩散的新一代商用级大语言模型。这些模型采用Transformer架构参数化，并训练为并行预测多个词元。本文详细介绍用于编程任务的首个扩散式大模型Mercury Coder，包含Mini和Small两个版本。独立评估显示，基于NVIDIA H100 GPU，Mercury Coder Mini与Small分别达到1109 tokens/sec和737 tokens/sec的吞吐率，在速度-质量前沿创下新纪录，平均比当前最优速度模型快10倍，同时保持相当的质量水平。我们在多语言、多场景代码基准上展示额外结果，并通过Copilot Arena的真实开发者验证：该模型目前在质量上排名第二，是整体最快模型。我们还开放了公共API（https://platform.inceptionlabs.ai/）和免费体验平台（https://chat.inceptionlabs.ai）。
原文摘要 · Abstract (English)
We present Mercury, a new generation of commercial-scale large language models (LLMs) based on diffusion. These models are parameterized via the Transformer architecture and trained to predict multiple tokens in parallel. In this report, we detail Mercury Coder, our first set of diffusion LLMs designed for coding applications. Currently, Mercury Coder comes in two sizes: Mini and Small. These models set a new state-of-the-art on the speed-quality frontier. Based on independent evaluations conducted by Artificial Analysis, Mercury Coder Mini and Mercury Coder Small achieve state-of-the-art throughputs of 1109 tokens/sec and 737 tokens/sec, respectively, on NVIDIA H100 GPUs and outperform speed-optimized frontier models by up to 10x on average while maintaining comparable quality. We discuss additional results on a variety of code benchmarks spanning multiple languages and use-cases as well as real-world validation by developers on Copilot Arena, where the model currently ranks second on quality and is the fastest model overall. We also release a public API at https://platform.inceptionlabs.ai/ and free playground at https://chat.inceptionlabs.ai
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。