RankGraph-2协同设计图学习全生命周期,提升百亿节点推荐系统召回率
RankGraph-2: Lifecycle Co-Design for Billion-Node Graph Learning in Recommendation
- 通过联合优化图构建、表征学习与实时服务,实现端到端协同设计
- 将边数从数万亿压缩至数百亿,服务计算成本降低83%,召回率提升2.1倍以上
- 适合大规模推荐系统研发者,尤其关注性能与可落地性的人群
在百亿节点规模的图检索中,图构建、表征学习和实时服务三个环节紧密耦合,但现有工作各自孤立处理。本文提出部署于Meta的RankGraph-2框架,针对基于相似性的检索(U2U2I和U2I2I)协同设计全生命周期。服务端依赖共学习的聚类索引避免在线KNN开销,推动索引共训练进入训练目标;训练阶段发现相似性检索可容忍预计算邻域,无需在线图基础设施,要求构建阶段输出自包含数据;构建还需支持小时级更新以保障商品覆盖。基于这些连锁需求,RankGraph-2通过流行度偏置校正的子采样将边数从数万亿降至数百亿,利用个性化PageRank预计算多跳邻域,并共学习残差量化聚类索引,使服务计算成本下降83%。该协同设计使简单架构在二分图上召回率是GAT+Deep Graph Infomax模型的3.8倍,在物品检索上比PyTorch-BigGraph高2.1倍。系统上线后带来最高+0.96%点击率和+2.75%转化率,已支撑20多个核心场景的检索上线。
原文摘要 · Abstract (English)
Graph-based retrieval at billion-node scale requires jointly solving three tightly coupled problems -- graph construction, representation learning, and real-time serving -- yet existing work addresses each in isolation. We present RankGraph-2, a framework deployed at Meta that co-designs all three lifecycle stages for similarity-based retrieval (U2U2I and U2I2I), where each stage's requirements shape the others. Serving requires a co-learned cluster index to avoid expensive online KNN -- this pushes index co-training into the training objective. Training benefits from the observation that similarity-based retrieval tolerates pre-computed neighborhoods, eliminating online graph infrastructure -- this requires construction to produce self-contained data. Construction must also support hour-level refresh for item coverage. Acting on these cascading requirements, RankGraph-2 reduces hundreds of trillions of edges to hundreds of billions via subsampling with popularity bias correction, pre-computes multi-hop neighborhoods via personalized PageRank, and co-learns a residual-quantization cluster index that reduces serving computational cost by 83%. This lifecycle co-design enables a simple architecture to achieve 3.8 x higher recall than a GAT + Deep Graph Infomax model on a bipartite graph and 2.1 x higher than PyTorch-BigGraph on item retrieval. RankGraph-2 delivers up to +0.96% CTR and +2.75% CVR, and has powered 20+ retrieval launches across major surfaces.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。