用大模型生成难负样本，让推荐系统更精准。
Real-Time Hard Negative Sampling via LLM-based Clustering for Large-Scale Two-Tower Retrieval
- 用大模型聚类生成难负样本，提升模型挑战性。
- 在大规模线上系统中显著降低热门内容偏差。
- 可实时部署，适合工业级推荐系统使用。
两塔模型广泛应用于大规模推荐系统，尤其在召回阶段。当前工业界训练两塔模型通常采用批内或批外负采样，但这些方法常产生易样本，模型易快速学习，难以真正提升性能。为此，本文提出一种基于大语言模型（LLM）的自监督难负样本采样技术，在训练过程中利用LLM从同一聚类中生成更具挑战性的负样本。通过让LLM学习媒体表征，确保生成的负样本更具区分度和信息量。该实时采样框架可无缝集成到生产模型中，支持数十亿数据点训练，计算开销极低。在公开数据集上的实验以及在大规模在线系统的部署均表明，该方法优于现有主流工业方案。此外，工业应用分析显示，该采样策略有助于打破推荐中的固有反馈循环，显著降低流行度偏差。
原文摘要 · Abstract (English)
The two-tower model has been widely used for large-scale recommendation systems, particularly in the retrieval stage. Industry standards for training two-tower models typically involve in-batch and/or out-of-batch negative sampling. However, these methods often produce easy negatives that models can quickly learn, failing to sufficiently challenge the model. To address this issue, a novel self-supervised hard negative sampling technique is proposed that leverages a large language model (LLM) to generate hard negatives from the same cluster during model training. By utilizing the LLM to learn media representations, the proposed approach ensures that the generated negatives are more challenging and informative. This real-time sampling framework is designed for seamless integration into production models, capable of handling billions of training data points with minimal computational complexity. Experiments on public datasets, along with deployment to a large-scale online system, demonstrate that the proposed negative sampling technique outperforms widely used industry methods. Furthermore, analysis in industrial applications reveals that this sampling method can help break inherent feedback loops in recommendations and significantly reduce popularity bias.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。