arXiv:2506.04165cs.LGcs.DS2025-06中稿 · TMLR May 2026被引 1

改进了快速选前K大值算法,提升加速器上的运行速度。

A Faster Generalized Two-Stage Approximate Top-K

  • 将分块选最大值改为每块选前K'个元素,优化输入规模。
  • 在相同召回率下减少分块数,显著降低第二阶段计算量。
  • 在TPUv5e上实现近10倍加速,适合机器学习推理场景。

本文研究Top-$K$选择问题,即从数组中找出最大的$K$个元素,该问题在众多机器学习算法中常见,常成为加速器性能瓶颈。针对此,Chern等(2022)提出一种快速两阶段近似Top-$K$算法:第一阶段将输入数组均分为若干块,每块选出最大值;第二阶段对这些最大值排序并返回前$K$个。本文将第一阶段推广为每块选出前$K'$个元素($1 \leq K' \leq K$),并给出随机分块下的期望召回率表达式。结果表明,适当增大$K'$并减少分块数,可在保持相同召回率的前提下更有效压缩第二阶段输入规模。此外,本文推导出原算法的期望召回率上界,其精度比原工作提高两倍。最后,我们在Cloud TPUv5e上实现了该算法,相较原方法获得约一个数量级的速度提升,且不损失召回率。

原文摘要 · Abstract (English)

We consider the Top-$K$ selection problem, which aims to identify the largest $K$ elements in an array. Top-$K$ selection arises in many machine learning algorithms and often becomes a bottleneck on accelerators, which are optimized for dense matrix multiplications. To address this problem, Chern et al. (2022) proposed a fast two-stage approximate Top-$K$ algorithm that: (i) partitions the input array into equal-sized chunks and selects the top-$1$ element from each partition; and (ii) sorts the resulting smaller subset and returns the top $K$ elements. In this paper, we generalize the first stage so that each partition selects the top $K'$ elements (for $1 \leq K' \leq K$). Our contributions include: (i) an expression for the expected recall of this generalized algorithm under random partitioning, and a demonstration that choosing $K' > 1$ with fewer partitions in the first stage more effectively reduces the input size to the second stage while maintaining the same expected recall as the original algorithm; (ii) a bound on the expected recall of the original algorithm as a function of the algorithm parameters that is provably tighter by a factor of $2$ than the bound reported by Chern et al. (2022); and (iii) an implementation of our algorithm on Cloud TPUv5e that achieves approximately an order of magnitude speedup over the original algorithm without sacrificing recall.

Top-K算法优化加速器

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。