arXiv:2410.17276cs.IRcs.LG2024-10被引 8

对比多种负采样方法,揭示其对推荐模型性能与热门商品偏见的影响。

Evaluating Performance and Bias of Negative Sampling in Large-Scale Sequential Recommendation Models

  • 在三种数据集上测试六种负采样方法,优化超参数并重复20次验证结果
  • 随机采样强化热门商品偏见,对头部商品表现最好但整体平衡性差
  • 基于热度的采样可提升中尾部商品表现,适合关注公平性的场景

大规模工业推荐系统需从数百万甚至数十亿商品中筛选最相关项。为高效训练,对每个正样本仅选取少量无关项作为负样本,帮助模型区分相关与不相关项。本文系统比较了随机、基于热度、批内、混合、自适应及自适应混合等六种负采样方法在现代序列推荐模型上的表现。实验涵盖超参数优化和三组基准数据集(含不同热度偏差)的20次重复,揭示采样方法与数据特征对关键性能指标的影响。发现平均性能指标常掩盖不同热度区间(头部、中部、尾部)间的不平衡问题。随机采样会加剧热度偏见,对头部商品表现最优;基于热度的采样(批内与全局热度)虽降低整体性能,但能实现更均衡的表现。本研究为大规模序列推荐中的负采样选择提供了实用权衡指南。代码、数据集、实验结果与超参数详见:https://github.com/apple/ml-negative-sampling。

原文摘要 · Abstract (English)

Large-scale industrial recommendation models predict the most relevant items from catalogs containing millions or billions of options. To train these models efficiently, a small set of irrelevant items (negative samples) is selected from the vast catalog for each relevant item (positive example), helping the model distinguish between relevant and irrelevant items. Choosing the right negative sampling method is a common challenge. We address this by implementing and comparing various negative sampling methods - random, popularity-based, in-batch, mixed, adaptive, and adaptive with mixed variants - on modern sequential recommendation models. Our experiments, including hyperparameter optimization and 20x repeats on three benchmark datasets with varying popularity biases, show how the choice of method and dataset characteristics impact key model performance metrics. We also reveal that average performance metrics often hide imbalances across popularity bands (head, mid, tail). We find that commonly used random negative sampling reinforces popularity bias and performs best for head items. Popularity-based methods (in-batch and global popularity negative sampling) can offer balanced performance at the cost of lower overall model performance results. Our study serves as a practical guide to the trade-offs in selecting a negative sampling method for large-scale sequential recommendation models. Code, datasets, experimental results and hyperparameters are available at: https://github.com/apple/ml-negative-sampling.

推荐系统负采样热度偏见序列推荐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。