arXiv:2410.16080cs.IR2024-10被引 10

提出高效融合多通道检索结果的新方法,提升推荐系统个性化与实时性。

Unleashing the Potential of Multi-Channel Fusion in Retrieval for Personalized Recommendations

  • 用优化权重融合多源检索结果,替代传统人工设计策略。
  • 在多个数据集上显著提升推荐效果,真实场景部署增益明显。
  • 适配个性化需求,可扩展至大规模工业级推荐系统。

推荐系统在应对现代数字服务中的信息过载问题中至关重要。核心挑战在于在严格延迟约束下,高效处理海量物品池并提供高度个性化的推荐。多阶段级联排序通过先用计算高效的检索方法覆盖多样用户兴趣,再由更精确的排序模型精炼结果来解决此问题。检索阶段常采用多通道检索,从不同候选生成器中获取互补的物品子集以最大化覆盖率。然而,直接传递所有检索结果会压垮下游排序器,需进行截断。尽管各检索方法不断进步,多通道融合——即高效合并多通道检索结果——仍缺乏系统研究。当前业界多依赖启发式和人工设计,常导致性能不佳。此外,传统基于梯度的方法(如SGD)因选择过程不可导而不适用。本文首次系统研究检索阶段的多通道融合,提出通过系统化优化各通道权重来实现高效融合。我们采用黑箱优化技术(如交叉熵法、贝叶斯优化)进行全局权重优化,并结合基于策略梯度的方法实现个性化融合。所提方法在多个数据集上均显著提升性能,真实场景部署中取得显著增益,为检索阶段的多通道融合提供了可扩展的优化方案。

原文摘要 · Abstract (English)

Recommender systems (RS) are pivotal in managing information overload in modern digital services. A key challenge in RS is efficiently processing vast item pools to deliver highly personalized recommendations under strict latency constraints. Multi-stage cascade ranking addresses this by employing computationally efficient retrieval methods to cover diverse user interests, followed by more precise ranking models to refine the results. In the retrieval stage, multi-channel retrieval is often used to generate distinct item subsets from different candidate generators, leveraging the complementary strengths of these methods to maximize coverage. However, forwarding all retrieved items overwhelms downstream rankers, necessitating truncation. Despite advancements in individual retrieval methods, multi-channel fusion, the process of efficiently merging multi-channel retrieval results, remains underexplored. We are the first to identify and systematically investigate multi-channel fusion in the retrieval stage. Current industry practices often rely on heuristic approaches and manual designs, which often lead to suboptimal performance. Moreover, traditional gradient-based methods like SGD are unsuitable for this task due to the non-differentiable nature of the selection process. In this paper, we explore advanced channel fusion strategies by assigning systematically optimized weights to each channel. We utilize black-box optimization techniques, including the Cross Entropy Method and Bayesian Optimization for global weight optimization, alongside policy gradient-based approaches for personalized merging. Our methods enhance both personalization and flexibility, achieving significant performance improvements across multiple datasets and yielding substantial gains in real-world deployments, offering a scalable solution for optimizing multi-channel fusion in retrieval.

推荐系统多通道融合个性化优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。