arXiv:2606.04603cs.IRcs.LG2026-06

让推荐系统更懂不确定性,提升长尾内容召回效果。

Distributional Approximate Nearest Neighbour Search for Uncertainty-Aware Retrieval

论文配图:Distributional Approximate Nearest Neighbour Search for Uncertainty-Aware Retrieval
图 1 · 摘自论文原文
  • 用采样多个嵌入表示替代单一嵌入,显式建模不确定性
  • 在不改动架构和索引的前提下,长尾内容召回率显著提升
  • 适合关注推荐公平性与多样性、追求真实用户反馈的研究者

近似最近邻搜索索引是现实推荐系统的核心,可在百万级商品库中实现实时候选检索。传统方法为每个用户和物品学习单一点估计嵌入,但这些嵌入基于稀疏交互数据,存在噪声且难以捕捉相关性的全部细节,导致系统偏向少数热门头部项目,牺牲了长尾、多样且意外的内容。我们提出 DINOSAUR(分布式近似最近邻搜索,用于不确定性感知检索):一种简单且兼容现有基础设施的框架,将嵌入不确定性融入候选生成。不再索引单一点估计,而是对每个物品采样 $S_i$ 个嵌入并构建扩展索引;查询时同样采样用户嵌入。这种双侧随机检索过程隐式对嵌入不确定性进行积分,无需修改模型架构或 ANN 索引基础设施。理论上,当不确定性消失时,DINOSAUR 恢复标准点估计检索,并揭示嵌入方差增大可扩大不确定物品可被检索的潜在空间区域。实验结果与预期一致,展示在离线召回率小幅下降的情况下,长尾覆盖范围大幅增加。

原文摘要 · Abstract (English)

Approximate Nearest Neighbour search indices form the backbone of real-world recommender systems, enabling real-time candidate retrieval over million-item catalogues. Typically, a single point estimate embedding is learnt for every user and every item. At serving time, the user embedding queries the index for relevant items. Since these representations are learnt from sparse interaction data, they are noisy and might fail to capture all the nuances that contribute to ``relevance'' -- ignoring the fundamental uncertainty that is inherent to them. The result is a retrieval pipeline that is systematically biased toward the small minority of popular head items with well-estimated embeddings, at the expense of the long-tail majority of niche, diverse, and serendipitous content. We propose DINOSAUR (Distributional Approximate Nearest Neighbour Search for Uncertainty-Aware Retrieval): a simple and infrastructure-compatible framework to incorporate embedding uncertainty into candidate generation. Rather than indexing point estimates, DINOSAUR samples $S_i$ embeddings per item and constructs an index on this augmented set. Analogously, at query time, a user embedding is sampled. This two-sided stochastic retrieval process implicitly marginalises over embedding uncertainty, without requiring changes to model architecture or ANN index infrastructure. On the analytical side, we show that DINOSAUR recovers standard point-estimate retrieval as uncertainty vanishes, and we characterise how increased embedding variance expands the regions of latent space in which uncertain items are retrievable. Reproducible empirical observations align with these expectations, showing large coverage gains with small losses in offline recall.

推荐系统不确定性建模长尾召回

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。