共享索引让推荐系统能零成本接入新内容,但牺牲了部分精度。
Keeping the Index Open: The Recommendation-Side Cost of Shared Search and Recommendation

- 用特征塔直接打分新内容,无需历史行为支持。
- 冷启动场景下召回率是专用模型的1.4倍,且无需专门训练。
- 适合追求快速上线新内容、容忍轻微精度损失的推荐系统。
共享搜索与推荐索引需仅凭特征评分新项目,因搜索无探索槽位。在覆盖两个界面的公开日志中,38.6%的待测查询-搜索曝光项为首次展示或访问。对用户冷启动场景,特征塔在99个采样负样本下表现接近热启动(Recall@20:0.9595 vs 0.9510),词法基线也达到相似效果,全量检查则统计上无显著差异。双编码器检索因此保持索引‘开放’,不同于依赖重训练的ID-softmax推荐器。我们以六种序列基线评估此开放性代价,每轮在修正目标上重新训练调优五次。浮点32位时间戳错误导致19.7%用户的留一法目标被错序。在MovieLens-1M上,热启动准确率比最强基线低5.2% Recall@20和11.4% NDCG@20;在MIND上差距缩小至0.8–3.6%,但模型排名第六。严格零泄露冷启动评估下,内容塔取得0.172±0.006 Recall@20,为最强专用方法(0.124±0.007)的1.4倍,三倍于无训练基线,且无需冷启动特训。精确全软标签训练使MIND-small的Recall@20提升54%,MovieLens-1M提升6.9%,但需每步重新计算全目录,24万项时耗尽加速器内存。近似最近邻搜索无法解释剩余差距,服务开销不随ID-softmax检索下降,历史窗口扫描解释了半数残差。大规模精确训练仍是未解难题。
原文摘要 · Abstract (English)
A shared search-and-recommendation index must score new items from features alone because search has no exploration slot. In a public log covering both surfaces over one catalog, $38.6\%$ of held-out query-search impressions show an item never previously shown or visited. For user-cold engagements, the feature-based tower serves this demand without measurable loss against $99$ sampled negatives ($0.9595$ Recall@20 versus $0.9510$ warm). A lexical baseline reaches similar parity, while a full-catalog check remains statistically undecided. Dual-encoder retrieval therefore keeps the index \emph{open} to new items, unlike an ID-softmax recommender that requires retraining. We price this openness on recommendation against six sequential baselines, each retrained and tuned through five rounds on corrected targets. A float32 timestamp bug had reordered leave-one-out targets for $19.7\%$ of users. On MovieLens-1M, warm accuracy trails the strongest retrained baseline by $5.2\%$ Recall@20 and $11.4\%$ NDCG@20. On MIND, the gap narrows to $0.8$--$3.6\%$ relative to the five strongest baselines, though the model ranks sixth of seven. Under strict zero-leakage cold-start evaluation, the content tower achieves $0.172 \pm 0.006$ Recall@20, $1.4\times$ the strongest retrained dedicated method ($0.124 \pm 0.007$) and $3\times$ a training-free floor, without cold-specific training. Exact full-softmax training raises Recall@20 by $54\%$ on MIND-small and $6.9\%$ on MovieLens-1M over sampled InfoNCE, but recomputes the full catalog each step and exhausts accelerator memory at $240$K items. Approximate nearest-neighbor search explains none of the remaining gap, serving cost does not regress against ID-softmax retrieval, and a history-window sweep explains half the post-recipe remainder. Exact-quality training at catalog scale remains the open problem.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。