让普通显卡跑6:8稀疏模型，速度翻倍还不丢精度。
SlideSparse: Fast and Flexible (2N-2):2N Structured Sparsity
- 用滑动窗口重构成2:4兼容块，实现无损稀疏加速。
- 在Qwen2.5-7B上达1.33倍实测提速，逼近理论极限。
- 适合想用便宜显卡跑大模型且不牺牲准确率的开发者。
NVIDIA的2:4稀疏张量核心能提升2倍吞吐量，但要求严格50%剪枝，导致大模型推理准确率暴跌（如Qwen3从54%降至15%）。更温和的(2N-2):2N稀疏模式（如6:8，25%剪枝）可保留精度，却无硬件支持，只能回退到密集计算，无法受益于稀疏性。本文提出SlideSparse，首个在通用GPU上实现(2N-2):2N稀疏模型加速的系统。通过滑动窗口分解，将任意(2N-2):2N权重块重构为N-1个重叠的2:4兼容窗口，无精度损失；激活重排与每令牌量化融合，开销极低。集成至vLLM后，在A100、H100、B200、RTX 4090、RTX 5080、DGX-spark等多款GPU上，适配多种精度（FP4、INT8、FP8、BF16、FP16）和模型家族（Llama、Qwen、BitNet），在计算密集型任务中，对Qwen2.5-7B的6:8稀疏模式实测提速1.33倍，接近理论上限4/3，证明(2N-2):2N是兼顾精度与加速的实用路径。代码已开源：https://github.com/bcacdwk/vllmbench。
原文摘要 · Abstract (English)
NVIDIA's 2:4 Sparse Tensor Cores deliver 2x throughput but demand strict 50% pruning -- a ratio that collapses LLM reasoning accuracy (Qwen3: 54% to 15%). Milder $(2N-2):2N$ patterns (e.g., 6:8, 25% pruning) preserve accuracy yet receive no hardware support, falling back to dense execution without any benefit from sparsity. We present SlideSparse, the first system to unlock Sparse Tensor Core acceleration for the $(2N-2):2N$ model family on commodity GPUs. Our Sliding Window Decomposition reconstructs any $(2N-2):2N$ weight block into $N-1$ overlapping 2:4-compliant windows without any accuracy loss; Activation Lifting fuses the corresponding activation rearrangement into per-token quantization at near-zero cost. Integrated into vLLM, SlideSparse is evaluated across various GPUs (A100, H100, B200, RTX 4090, RTX 5080, DGX-spark), precisions (FP4, INT8, FP8, BF16, FP16), and model families (Llama, Qwen, BitNet). On compute-bound workloads, the measured speedup ratio (1.33x) approaches the theoretical upper-bound $N/(N-1)=4/3$ at 6:8 weight sparsity in Qwen2.5-7B, establishing $(2N-2):2N$ as a practical path to accuracy-preserving LLM acceleration. Code available at https://github.com/bcacdwk/vllmbench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。