通过预测性稀疏加速大模型预填充，提升长文本推理效率。
Fast Forward: Accelerating LLM Prefill with Predictive FFN Sparsity
- 按块预测重要神经元，实现上下文感知的前馈网络稀疏化
- 在50%稀疏度下提速1.45倍，准确率损失低于6%
- 适合资源受限设备上的长序列推理场景
大语言模型推理中的预填充阶段是长上下文任务的主要计算瓶颈。在1K至16K token的中短上下文长度下，前馈网络（FFN）占用了绝大部分计算量（FLOPs）。现有针对自回归解码设计的FFN稀疏化方法无法利用预填充阶段的并行性，且常导致精度下降。为此，本文提出FastForward，一种基于预测的稀疏化框架，通过块级、上下文感知的FFN稀疏策略加速预填充。该方法结合轻量级专家预测器（每块选择高重要性神经元）、误差补偿网络（修正稀疏引入的误差）和逐层稀疏调度器（根据词元混合重要性分配算力）。在LLaMA与Qwen系列模型（最高80亿参数）上，FastForward在50% FFN稀疏度下相较密集基线实现高达1.45倍的计算瓶颈加速，LongBench测试中准确率损失小于6%，显著降低首次生成时间（TTFT），适用于受限硬件上的高效长上下文推理。
原文摘要 · Abstract (English)
The prefill stage of large language model (LLM) inference is a key computational bottleneck for long-context workloads. At short-to-moderate context lengths (1K--16K tokens), Feed-Forward Networks (FFNs) dominate this cost, accounting for most of the total FLOPs. Existing FFN sparsification methods, designed for autoregressive decoding, fail to exploit the prefill stage's parallelism and often degrade accuracy. To address this, we introduce FastForward, a predictive sparsity framework that accelerates LLM prefill through block-wise, context-aware FFN sparsity. FastForward combines (1) a lightweight expert predictor to select high-importance neurons per block, (2) an error compensation network to correct sparsity-induced errors, and (3) a layer-wise sparsity scheduler to allocate compute based on token-mixing importance. Across LLaMA and Qwen models up to 8B parameters, FastForward delivers up to 1.45$\times$ compute-bound speedup at 50% FFN sparsity with $<$ 6% accuracy loss compared to the dense baseline on LongBench, substantially reducing Time-to-First-Token (TTFT) for efficient, long-context LLM inference on constrained hardware.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。