改进了快速选前K大值算法,提升加速器上的运行速度。
A Faster Generalized Two-Stage Approximate Top-K
- 将分块选最大值改为每块选前K'个元素,优化输入规模。
- 在相同召回率下减少分块数,显著降低第二阶段计算量。
- 在TPUv5e上实现近10倍加速,适合机器学习推理场景。
本文研究Top-$K$选择问题,即从数组中找出最大的$K$个元素,该问题在众多机器学习算法中常见,常成为加速器性能瓶颈。针对此,Chern等(2022)提出一种快速两阶段近似Top-$K$算法:第一阶段将输入数组均分为若干块,每块选出最大值;第二阶段对这些最大值排序并返回前$K$个。本文将第一阶段推广为每块选出前$K'$个元素($1 \leq K' \leq K$),并给出随机分块下的期望召回率表达式。结果表明,适当增大$K'$并减少分块数,可在保持相同召回率的前提下更有效压缩第二阶段输入规模。此外,本文推导出原算法的期望召回率上界,其精度比原工作提高两倍。最后,我们在Cloud TPUv5e上实现了该算法,相较原方法获得约一个数量级的速度提升,且不损失召回率。
原文摘要 · Abstract (English)
We consider the Top-$K$ selection problem, which aims to identify the largest $K$ elements in an array. Top-$K$ selection arises in many machine learning algorithms and often becomes a bottleneck on accelerators, which are optimized for dense matrix multiplications. To address this problem, Chern et al. (2022) proposed a fast two-stage approximate Top-$K$ algorithm that: (i) partitions the input array into equal-sized chunks and selects the top-$1$ element from each partition; and (ii) sorts the resulting smaller subset and returns the top $K$ elements. In this paper, we generalize the first stage so that each partition selects the top $K'$ elements (for $1 \leq K' \leq K$). Our contributions include: (i) an expression for the expected recall of this generalized algorithm under random partitioning, and a demonstration that choosing $K' > 1$ with fewer partitions in the first stage more effectively reduces the input size to the second stage while maintaining the same expected recall as the original algorithm; (ii) a bound on the expected recall of the original algorithm as a function of the algorithm parameters that is provably tighter by a factor of $2$ than the bound reported by Chern et al. (2022); and (iii) an implementation of our algorithm on Cloud TPUv5e that achieves approximately an order of magnitude speedup over the original algorithm without sacrificing recall.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。