PRISM通过拆分计算路径，让大模型推理更快更省资源。
PRISM: Parametrically Refactoring Inference for Speculative Sampling Draft Models
- 将每步预测计算拆分到不同参数集，解耦模型能力与推理开销。
- 在保持低延迟前提下，实现更长的接受长度，端到端加速超2.6倍。
- 适合追求高吞吐、低延迟推理的LLM系统开发者和研究者。
大型语言模型受限于自回归特性，推理速度缓慢。推测解码方法成为加速推理的有前景方案，受到系统与人工智能研究界的广泛关注。近期，为提升推测质量，学界趋向采用参数量更大的推测模型，但这带来了显著的计算开销。现有工作尝试在预测准确率与计算延迟间权衡，而我们通过架构创新，从根本上解决这一矛盾。提出PRISM，将每步预测的计算分解至不同的参数集合，重构推测模型的计算路径，成功实现模型容量与推理成本的解耦。大量实验表明，PRISM优于所有现有推测架构，在保持极低推测延迟的同时，实现卓越的接受长度，带来优异的端到端加速效果。重新审视缩放定律发现，相较于其他推测架构，PRISM在数据量扩大时具有更强的可扩展性。通过严格公平对比，证明其使已高度优化的推理引擎解码吞吐量提升超过2.6倍。
原文摘要 · Abstract (English)
Large Language Models (LLMs), constrained by their auto-regressive nature, suffer from slow decoding. Speculative decoding methods have emerged as a promising solution to accelerate LLM decoding, attracting attention from both systems and AI research communities. Recently, the pursuit of better draft quality has driven a trend toward parametrically larger draft models, which inevitably introduces substantial computational overhead. While existing work attempts to balance the trade-off between prediction accuracy and compute latency, we address this fundamental dilemma through architectural innovation. We propose PRISM, which disaggregates the computation of each predictive step across different parameter sets, refactoring the computational pathways of draft models to successfully decouple model capacity from inference cost. Through extensive experiments, we demonstrate that PRISM outperforms all existing draft architectures, achieving exceptional acceptance lengths while maintaining minimal draft latency for superior end-to-end speedup. We also re-examine scaling laws with PRISM, revealing that PRISM scales more effectively with expanding data volumes than other draft architectures. Through rigorous and fair comparison, we show that PRISM boosts the decoding throughput of an already highly optimized inference engine by more than 2.6x.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。