用哈希嵌入和时间采样提升百万级社交图的推荐效率
Scaling Graph Neural Networks for Friend Recommendation: Multi-Hash User Embeddings and Temporal Neighbor Sampling
- 用多哈希技术压缩用户嵌入表,节省98%存储空间
- 引入时间排序的二分查找采样,降低邻居搜索开销
- 在1.94亿用户图上实现推荐量提升16%,适合大规模社交系统
好友推荐本质上是图结构问题:潜在连接的相关性依赖于多跳社交上下文,而非仅靠用户属性。但在拥有1.94亿用户、280亿边的生产级社交图上部署消息传递GNN面临建模与系统双重挑战。本文提出一个可扩展的端到端GNN排序系统,聚焦两大关键设计:多哈希用户嵌入与时间邻居采样。多哈希嵌入虽常用于高基数特征,但工业级GNN系统通常要么忽略可训练ID,要么接受全量嵌入表(本图超200GB)。我们将其作为主要节点表示,使嵌入表大小减少超过98%,同时保持排名质量。时间邻居采样在理论上成熟,但现有实现需扫描完整邻接表,对有数万好友的用户不可行。我们采用时间戳排序的压缩稀疏行存储(CSR)结合二分查找,将每节点采样复杂度从O(deg(v) + k)降至O(log(deg(v)) + k)。除上述组件外,该组合在实际场景中可扩展且产生显著影响。在194M用户、28B边的图上,离线消融实验分离出各设计贡献。在线A/B测试显示,相较强基线,系统使推荐带来的好友添加量提升16%,独特加友人数提升11.5%。我们公开了支持分布式训练与推理的大规模时序图框架。
原文摘要 · Abstract (English)
Friend recommendation is inherently graph-structured: the relevance of a potential connection depends on multi-hop social context rather than user attributes alone. However, deploying message-passing GNNs on a production-scale social graph with hundreds of millions of users and tens of billions of edges requires addressing numerous modeling and systems challenges. We present a scalable end-to-end GNN ranking system for production social graphs, focusing on two design choices that are critical in this setting: multi-hash ID embeddings and temporal neighbor sampling. Multi-hash embeddings are common for high-cardinality features, but industrial GNN systems typically either ignore trainable IDs or accept full embedding tables, exceeding 200 GB for our graph. We integrate multi-hash as the primary node representation, reducing the ID-embedding table size by more than 98 percent while preserving ranking quality. Temporal neighbor sampling is well understood in principle, but existing implementations scan full adjacency lists, which is a non-starter for users with tens of thousands of friends. We implement timestamp-sorted CSR storage with binary search, reducing the per-node temporal sampling cost from $O(deg(v) + k)$ to $O(\log(deg(v)) + k)$. Beyond these components, we show that this combination scales and yields measurable production impact. On a graph with 194M users and 28B edges, offline ablations isolate each design choice's contribution. In an online A/B test, our system increases friend additions from recommendations by 16 percent and unique friend adders by 11.5 percent over a strong production baseline. We release our framework for distributed training and inference on large temporal graphs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。