GPU加速稀疏检索，实现毫秒级精准排序，速度提升235倍。
GPUSparse: GPU-Accelerated Learned Sparse Retrieval with Parallel Inverted Indices
- GPU并行倒排索引，优化数据布局与线程访问
- 批量散列累加算法，支持数百查询并发处理
- 精确检索达787 QPS，适合大规模实时场景
学习型稀疏检索模型如SPLADE在保持可解释性的同时，达到与稠密模型相当的检索效果。然而，推理阶段的评分仍依赖于受CPU限制的倒排索引遍历算法（如WAND、Block-Max WAND），成为大规模实时服务的瓶颈。本文提出GPUSparse，一种基于GPU的精确学习型稀疏检索系统，包含：(1) 块对齐、线程束合并的GPU并行倒排索引；(2) 支持批量查询的散列累加评分算法；(3) 融合Triton内核，分析工作效率与硬件利用率之间的权衡。在MS MARCO段落排名任务（880万段落）上，使用真实SPLADE嵌入，GPUSparse在三个小数位精度下与CPU精确评分一致（MRR@10=0.383，与Pyserini SPLADE相同；Recall@1000≥0.999，与密集矩阵乘法差异仅来自浮点数平局判定）。相比Pyserini CPU，查询延迟从298ms降至1.27ms，提速235倍。相较当前最快CPU系统Seismic（牺牲25%召回率换取速度，R@1000=0.738 vs. 0.983），GPUSparse在完整880万文档集上以批处理500达到787 QPS吞吐，单次查询仅需1.3ms。其文档并行内核达到H100峰值显存带宽的62.6%，揭示了工作效率与带宽效率间的根本权衡。将稀疏评分重构为倒排索引上的散列累加操作，虽与SPARe迭代模式共享思想，但本工作的融合内核实现比忠实复现的SPARe迭代版本快23–270倍。
原文摘要 · Abstract (English)
Learned sparse retrieval models such as SPLADE achieve retrieval quality competitive with dense models while preserving the interpretability and exact-match advantages of sparse representations. However, inference-time scoring still relies on CPU-bound inverted index traversal algorithms (WAND, Block-Max WAND), creating a fundamental bottleneck for real-time serving at scale. We present GPUSparse, a system for GPU-accelerated exact learned sparse retrieval that introduces: (1) a GPU-parallel inverted index with block-aligned, warp-coalesced posting lists; (2) a batched scatter-add scoring algorithm that processes hundreds of queries simultaneously; and (3) fused Triton kernels with an analysis of the tradeoff between work-efficiency and hardware utilization. On MS MARCO passage ranking (8.8M passages) with real SPLADE embeddings, GPUSparse matches CPU exact scoring to three decimals (MRR@10=0.383, equal to Pyserini SPLADE at this precision; Recall@1000>=0.999 vs. dense matmul, the residual from floating-point tie-breaking) while providing a 235x speedup over Pyserini CPU at 8.8M documents (1.27ms vs. 298ms per query). Compared to Seismic (the fastest CPU sparse retrieval system), which trades 25% recall for speed (R@1000=0.738 vs. 0.983 exact), GPUSparse achieves exact scoring at 787 QPS throughput (batch 500) on the full 8.8M collection, with 1.3ms per query. Our document-parallel kernel reaches 62.6% of H100 peak HBM bandwidth, revealing a fundamental work-efficiency vs. bandwidth-efficiency tradeoff in GPU sparse retrieval. The reformulation of sparse scoring as scatter-add over an inverted index is shared with SPARe's iterative mode; our contribution is its fused-kernel realization, which we measure to be 23-270x faster than a faithful SPARe iterative reimplementation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。