为扩散语言模型设计专用加速器,解决传统NPU不兼容问题。
NPU Design for Diffusion Language Model Inference
- 针对扩散模型特点设计新指令集与执行架构。
- 提出新型量化方法,支持动态块缓存更新,精度损失小。
- 适合研究扩散模型推理或芯片设计的开发者使用。
基于扩散的大型语言模型(dLLMs)在推理机制上与传统自回归模型有本质差异:采用双向注意力、分块刷新键值缓存、跨步骤复用,以及非以GEMM为中心的采样阶段。这些特性使现有NPU难以支持,尤其在高计算密度的top-k采样和分块缓存更新场景下,需新的指令集架构(ISA)与内存层次结构。此外,分块扩散缓存打破了自回归模型假设的追加式写入范式,传统静态激活分布的量化方案无法适应迭代式分块优化带来的分布变化。本文首次提出专为dLLMs设计的NPU加速器,包含面向dLLM的ISA与编译器、硬件优化的执行模型、创新的块自适应在线平滑(BAOS)量化方法,并完成7nm工艺的完整RTL实现。为验证设计,构建了三路径仿真框架,包含分析型、周期精确型与精度仿真器,并与真实硬件交叉验证。整个NPU栈(含ISA、仿真工具与量化软件)将在论文接受后开源。
原文摘要 · Abstract (English)
Diffusion-based LLMs (dLLMs) fundamentally depart from traditional autoregressive (AR) LLM inference: they leverage bidirectional attention, block-wise KV cache refreshing, cross-step reuse, and a non-GEMM-centric sampling phase. These characteristics make current dLLMs incompatible with most existing NPUs, as their inference patterns, in particular the reduction-heavy, top-$k$-driven sampling stage, demand new ISA and memory hierarchy support beyond that of AR accelerators. In addition, the blocked diffusion KV cache breaks from the append-only paradigm assumed by AR NPUs, and conventional AR-derived KV quantization schemes were designed for static activation distributions and do not account for the step-wise distribution shifts introduced by iterative block-wise refinement in dLLMs. In this paper, we introduce the first NPU accelerator specifically designed for dLLMs. It delivers: a dLLM-oriented ISA and compiler; a hardware-optimized execution model for both the transformer inference and diffusion sampling used in dLLMs; a novel Block-Adaptive Online Smoothing (BAOS) for quantizing KV cache in dLLMs; and a complete RTL implementation synthesized in 7nm. To evaluate and validate our design, we introduce a tri-path simulation framework that comprises analytical, cycle-accurate, and accuracy simulators, together with cross-validations against physical hardware. The full NPU stack, including ISA, simulation tools, and quantization software, will be open-sourced upon acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。