arXiv:2511.15898cs.LG2025-11被引 2

提出高效多草稿采样算法,实现90%采纳率且延迟低于100毫秒。

Global Resolution: Optimal Multi-Draft Speculative Sampling via Convex Minimization

  • 将最优传输问题转化为凸优化,仅需最多V个变量求解。
  • 在顶-k采样下实现90%令牌采纳率,每生成一个词延迟低于100毫秒。
  • 适用于对推理速度敏感的LLM部署场景,如实时对话系统。

推测采样通过使用廉价草稿模型预测候选词并用验证准则决定是否接受,可降低自回归解码延迟而不损失质量。为提升采纳率与效率,近年研究扩展至多草稿机制:每步生成n个草稿词,并基于这些词定义条件分布进行验证。当该准则最大化接受任一草稿词的概率时,称为最优传输(OT)。然而求解OT困难,因涉及超过$V^n$变量的线性规划(OTLP),其中$V$为词汇量。已有理论工作将其重述为重要性采样或子集选择形式,但本文证明这些等价于指数级放大的松弛型OTLP,仍不可行。我们反向设计子集选择,将OTLP重构为最大流问题,并利用拟阵理论将其简化为最多$V$变量的凸优化。由此提出一种可在独立同分布草稿采样下实现任意精度的最优n-草稿推测采样算法。实验测量了不同n和top-k设置下的采纳率与运行时间,结果首次实现90%采纳率且每生成一个词的开销低于100毫秒,同时与目标模型分布偏差极小。

原文摘要 · Abstract (English)

Speculative sampling reduces the latency of autoregressive decoding for target model LLMs without sacrificing inference quality, by using a cheap draft model to suggest a candidate token and a verification criterion to accept or resample this token. To improve acceptance and decoding efficiency, recent work has explored the multi-draft extension, where at each step $n$ draft tokens are generated, and the verification criterion is a distribution conditioned on these. When this criterion maximizes the probability of accepting some draft token, it is called the optimal transport (OT). However, finding the OT is difficult, as it is the solution of a linear program (OTLP) in over $V^n$ variables, with $V$ being the vocabulary size. Two recent theoretical works have reframed the OTLP in terms of importance sampling or subset selection. In this work, we prove that these formulations are equivalent to an exponentially large relaxed OTLP, so it remains infeasible to solve. Then, we reverse engineer subset selection to formulate the OTLP as a max-flow problem. With a novel application of polymatroid theory, we reduce the exponentially large OTLP to a convex optimization problem in at most $V$ variables. This allows us to devise an algorithm for optimal $n$-draft speculative sampling when the $n$ tokens are chosen i.i.d. from a single draft model, which can be tuned to arbitrary accuracy. Finally, we measure acceptance rates and algorithm runtimes for various $n$ and top-$k$ draft sampling settings. Our findings give the first multi-draft algorithm with 90% acceptance and under 100 ms of overhead per generated token with negligible deviation from the target model distribution.

推理加速多草稿采样凸优化大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。