arXiv:2502.11490cs.LGcs.DC2025-02

Baidu提出GPU加速的多关系图检索框架,提升超大规模推荐系统的效率与准确率。

GPU-accelerated Multi-relational Parallel Graph Retrieval for Web-scale Recommendations

  • 用多目标优化统一用户行为建模,提升路径搜索能力
  • 通过分层并行图搜索实现每秒超亿级请求吞吐
  • 适合高并发、海量数据场景下的推荐系统研发者

Web推荐系统需从百亿级候选中高效筛选相关项目,Baidu采用深度神经网络和基于图的近似最近邻搜索(ANNS)以平衡效果与效率。但现有方法在用户-物品关系理解上存在行为拆解问题,且在大规模图搜索中因遍历策略不佳与GPU计算瓶颈导致效率低下。为此,本文提出GPU加速的多关系并行图检索框架(GMP-GR)。首先,设计多关系用户-物品相关性度量学习方法,通过多目标优化统一多样化用户行为,并引入自协方差损失增强路径寻优能力;其次,构建层次化并行图ANNS,实现大尺度物品图上的广度-深度平衡搜索,并通过GPU自适应聚合有效处理不规则神经计算。此外,集成部署优化策略。大量实验表明GMP-GR在准确率与效率上均显著优于现有方法。已在Baidu超过20个应用上线,支撑数亿用户,单系统吞吐超1亿请求/秒。

原文摘要 · Abstract (English)

Web recommendations provide personalized items from massive catalogs for users, which rely heavily on retrieval stages to trade off the effectiveness and efficiency of selecting a small relevant set from billion-scale candidates in online digital platforms. As one of the largest Chinese search engine and news feed providers, Baidu resorts to Deep Neural Network (DNN) and graph-based Approximate Nearest Neighbor Search (ANNS) algorithms for accurate relevance estimation and efficient search for relevant items. However, current retrieval at Baidu fails in comprehensive user-item relational understanding due to dissected interaction modeling, and performs inefficiently in large-scale graph-based ANNS because of suboptimal traversal navigation and the GPU computational bottleneck under high concurrency. To this end, we propose a GPU-accelerated Multi-relational Parallel Graph Retrieval (GMP-GR) framework to achieve effective yet efficient retrieval in web-scale recommendations. First, we propose a multi-relational user-item relevance metric learning method that unifies diverse user behaviors through multi-objective optimization and employs a self-covariant loss to enhance pathfinding performance. Second, we develop a hierarchical parallel graph-based ANNS to boost graph retrieval throughput, which conducts breadth-depth-balanced searches on a large-scale item graph and cost-effectively handles irregular neural computation via adaptive aggregation on GPUs. In addition, we integrate system optimization strategies in the deployment of GMP-GR in Baidu. Extensive experiments demonstrate the superiority of GMP-GR in retrieval accuracy and efficiency. Deployed across more than twenty applications at Baidu, GMP-GR serves hundreds of millions of users with a throughput exceeding one hundred million requests per second.

推荐系统图检索GPU加速大规模推荐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。