提出线性时间可微顶K选择方法,提升推荐系统训练效率与性能。
Differentiable Fast Top-K Selection for Large-Scale Recommendation
- 通过闭式解避免排序,实现O(n)时间复杂度的可微顶K操作。
- 在真实工业系统中实现+1.77%收入提升,训练效率显著提高。
- 首个用于推荐系统的可微顶K算子,适合大规模在线学习场景。
级联排序是大规模信息检索中选取前K个项目的主流范式,但其顶K操作不可微,阻碍端到端训练。现有方法包括基于学习排序(如LambdaLoss)的指标优化,存在目标不一致问题;或基于可微排序的方法(如ARF、LCRON),依赖软排列矩阵导致梯度冲突。本文提出DFTopK,一种新型可微顶K算子,直接构建顶K选择的可微近似,绕过软排列矩阵。相比现有最优方法(如LapSum)需依赖排序实现阈值求解的O(n log n)复杂度,DFTopK通过放松归一化约束,获得闭式解并完全避免排序,实现理论最优的O(n)时间复杂度。该方法还规避了可微排序方法固有的梯度冲突。在公开基准RecFLow和工业系统上评估显示,DFTopK显著提升训练效率,支持更大规模样本训练。在线A/B测试中,相同计算开销下带来+1.77%收入增长。据我们所知,这是首个将可微顶K算子引入推荐系统的工作,也是首个实现顶K选择线性时间复杂度的方案。代码已开源,促进学术与产业研究。
原文摘要 · Abstract (English)
Cascade ranking is a widely adopted paradigm in large-scale information retrieval systems for Top-K item selection. However, the Top-K operator is non-differentiable, hindering end-to-end training. Existing methods include Learning-to-Rank approaches (e.g., LambdaLoss), which optimize ranking metrics like NDCG and suffer from objective misalignment, and differentiable sorting-based methods (e.g., ARF, LCRON), which relax permutation matrices for direct Top-K optimization but introduce gradient conflicts through matrix aggregation. A promising alternative is to directly construct a differentiable approximation of the Top-K selection operator, bypassing the use of soft permutation matrices. However, even state-of-the-art differentiable Top-K operator (e.g., LapSum) require $O(n \log n)$ complexity due to their dependence on sorting for solving the threshold. Thus, we propose DFTopK, a novel differentiable Top-K operator achieving optimal $O(n)$ time complexity. By relaxing normalization constraints, DFTopK admits a closed-form solution and avoids sorting. DFTopK also avoids the gradient conflicts inherent in differentiable sorting-based methods. We evaluate DFTopK on both the public benchmark RecFLow and an industrial system. Experimental results show that DFTopK significantly improves training efficiency while achieving superior performance, which enables us to scale up training samples more efficiently. In the online A/B test, DFTopK yielded a +1.77% revenue lift with the same computational budget compared to the baseline. To the best of our knowledge, this work is the first to introduce differentiable Top-K operators into recommendation systems and the first to achieve theoretically optimal linear-time complexity for Top-K selection. We have open-sourced our implementation to facilitate future research in both academia and industry.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。