arXiv:2412.00714cs.IR2024-12被引 38

探索大模型推荐系统规模效应,揭示性能提升规律。

Scaling New Frontiers: Insights into Large Recommendation Models

  • 对比不同架构,分析大推荐模型的缩放规律。
  • 验证HSTU模型在复杂行为建模中显著优于传统模型。
  • 首次系统评估其在排序任务中的效果,为未来方向提供参考。

推荐系统在各类应用中对数据过滤与信息检索至关重要。近年来,嵌入表规模已扩展至数十TB,但传统推荐模型的网络参数增长停滞在数千万量级,限制了进一步收益。受大语言模型启发,新方法通过创新结构扩大网络参数,实现持续性能提升。典型代表是Meta的生成式推荐模型HSTU,其参数量达数千亿级别,已在线上实验中取得显著成效。本文旨在深化对缩放规律的理解,系统评估大推荐模型:首先考察不同骨干架构下的缩放特性;其次开展全面消融实验,探究规律来源;进一步评估HSTU在复杂用户行为建模任务中的表现,并首次分析其在排序任务中的有效性。最后,提出未来研究方向。补充材料详见GitHub:https://github.com/USTC-StarTeam/Large-Recommendation-Models。

原文摘要 · Abstract (English)

Recommendation systems are essential for filtering data and retrieving relevant information across various applications. Recent advancements have seen these systems incorporate increasingly large embedding tables, scaling up to tens of terabytes for industrial use. However, the expansion of network parameters in traditional recommendation models has plateaued at tens of millions, limiting further benefits from increased embedding parameters. Inspired by the success of large language models (LLMs), a new approach has emerged that scales network parameters using innovative structures, enabling continued performance improvements. A significant development in this area is Meta's generative recommendation model HSTU, which illustrates the scaling laws of recommendation systems by expanding parameters to thousands of billions. This new paradigm has achieved substantial performance gains in online experiments. In this paper, we aim to enhance the understanding of scaling laws by conducting comprehensive evaluations of large recommendation models. Firstly, we investigate the scaling laws across different backbone architectures of the large recommendation models. Secondly, we conduct comprehensive ablation studies to explore the origins of these scaling laws. We then further assess the performance of HSTU, as the representative of large recommendation models, on complex user behavior modeling tasks to evaluate its applicability. Notably, we also analyze its effectiveness in ranking tasks for the first time. Finally, we offer insights into future directions for large recommendation models. Supplementary materials for our research are available on GitHub at https://github.com/USTC-StarTeam/Large-Recommendation-Models.

推荐系统大模型缩放定律

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。