通过检测生成过程中的表示漂移，提升扩散型大模型的推理效率与质量。
Polestar: Drift-Aware Cache Calibration and Token Commitment for Efficient Inference of Diffusion LLMs
- 利用令牌表示漂移统一解决缓存复用与提交时机问题
- 最高提升10.73%准确率，吞吐量达基线3.7倍
- 适合追求高效高质推理的扩散语言模型应用
扩散大语言模型（dLLMs）的推理效率受两大挑战制约：双向注意力阻碍了键值缓存（KV-cache）的有效复用；而使用静态置信度阈值提高解码并行性则可能损害生成质量。我们观察到，这两类问题均源于同一现象：随着解码进行，令牌通过双向注意力不断整合上下文，其表示在解码步骤间持续漂移。基于此洞察，我们提出Polestar——一种无需训练的推理框架，以令牌表示漂移作为统一信号，协同解决上述两个问题。Polestar包含两部分：Polestar-Cache通过检测漂移识别过时缓存位置，实施稀疏刷新以实现高效复用；Polestar-Commit则通过捕捉剧烈漂移事件，可靠判断可提交的令牌。在多个dLLM系列的数学与编码基准测试中，Polestar在准确率-吞吐率权衡曲线上达到新最优，最高实现10.73%准确率提升、3.7倍吞吐量增长，并保持3.67个令牌/前向传播的高并行度。
原文摘要 · Abstract (English)
The inference efficiency of diffusion large language models (dLLMs) is constrained by two challenges: bidirectional attention precludes efficient KV-cache reuse, while increasing decoding parallelism with static confidence thresholds can compromise generation quality. We observe that both challenges arise from a shared phenomenon: as tokens are decoded, their contextual integration through bidirectional attention causes token representations to drift (evolve) across decoding steps. This insight motivates Polestar, a training-free inference framework that uses token representation drift as a unified signal to jointly address both challenges. Polestar comprises two components: Polestar-Cache, which identifies stale KV-cache positions via drift and performs sparse KV-cache refreshes to enable efficient reuse, and Polestar-Commit, which detects sharp drift events to reliably identify commit-ready tokens. Across mathematics and coding benchmarks on several dLLM families, Polestar sets a new state of the art on the accuracy-throughput Pareto frontier, achieving up to 10.73% accuracy improvement, up to 3.7x higher throughput, and high decoding parallelism of 3.67 tokens per forward pass over existing baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。