MESH统一异构内容检索,显著提升冷门和新内容的召回效果。
MESH: Scaling Up Retrieval with Heterogeneous Content Unification

- 通过分域特征空间与门控偏差修正,减少稀疏内容与高频特征干扰。
- 新内容的幂律缩放指数提升14倍,线上实验新内容分享率提升5.5%。
- 适合大规模推荐系统优化,尤其关注冷启动和长尾内容场景。
大规模检索优化依赖于跨多样内容层级高效召回候选。然而,为捕捉新内容和长尾内容,现代系统通常采用多个专用检索模型,导致架构复杂。这源于异构检索系统的根本挑战——异构缩放偏差:模型容量提升对不同内容层级的增益不均等。为此,我们提出MESH统一检索缩放框架,通过模块化架构与门控偏差修正缓解该偏差。将特征空间划分为独立域,引入结构归纳偏置,降低稀疏项信号与高频互动特征间的干扰。受保护的梯度路径使稀疏内容缩放行为显著改善,实证显示新内容的幂律缩放指数提升14倍。在Pinterest相关图片推荐平台(百亿规模)的在线评估中,新内容分享率提升5.5%,转化漏斗效率提高55%,用户留存率提升0.46%。异步服务策略实现生产可用性,系统吞吐量提升2.87倍。结果表明,MESH是整合碎片化检索基础设施、构建更可扩展、生态感知后端的有前景范式。
原文摘要 · Abstract (English)
Optimizing large-scale retrieval hinges on the ability to efficiently surface candidates across diverse content tiers. However, to capture segments such as fresh and long-tail content, modern systems typically resort to a fragmented "zoo" of specialized retrieval models. This operational complexity is attributed to a fundamental challenge in heterogeneous retrieval systems, the Scaling Bias of Heterogeneity, where model capacity gains do not apply equally across diverse content tiers. To bridge this gap, we propose MESH as a unified retrieval scaling framework that mitigates this bias through a modularized architecture integrated with gated bias correction. By partitioning the feature space into independent domains, MESH enforces a structural inductive bias that reduces interference between sparse-item signals and high-frequency engagement features. This protected gradient path leads to improved scaling behavior for sparse content, empirically validated by a 14 times improvement in the power-law scaling exponent for fresh items. In online evaluations on Pinterest's Related Pins platform, a billion scale item-to-item recommendation system, these improvements translate into a +5.5% lift in fresh-item repins, alongside with 55% improvement in funnel efficiency and +0.46% improvement in user retention. Finally, our asynchronous serving strategy ensures production viability by delivering a 2.87 times improvement in system throughput. Our findings suggest MESH as a promising paradigm for consolidating fragmented retrieval infrastructures into more scalable and ecosystem-aware backbones.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。