发现推荐系统生成式召回存在可量化的缩放定律
OnePiece: The Great Route to Generative Recommendation -- A Case Study from Tencent Algorithm Competition
- 构建统一编码解码框架，同时验证两种推荐范式
- 两种方法损失均严格符合幂律缩放（R²>0.9）
- 为大模型时代推荐系统设计提供理论依据
近年来，OpenAI的缩放定律表明，通过扩大模型参数规模，基于下一个词预测的神经语言建模能显著提升智能水平。在推荐系统中，检索阶段也遵循类似‘下一个项目预测’范式，从全局物品集中召回数百个候选项，因此生成式推荐通常特指该阶段（不包括树形方法）。这引发一个哲学问题：在无真实下一物品的情况下，生成式推荐是否也存在潜在的缩放定律？回顾可知，生成式推荐存在两种技术范式：(1) 基于近似最近邻（ANN）的框架，利用压缩用户嵌入在嵌入空间中检索最近物品，如Kuaiformer；(2) 自回归框架，采用束搜索从全空间解码物品，如OneRec。本文设计了一个统一的编码器-解码器框架，同时验证两者的缩放定律。实证结果表明，在统一架构下，两者损失均严格遵循幂律缩放规律（R² > 0.9）。
原文摘要 · Abstract (English)
In past years, the OpenAI's Scaling-Laws shows the amazing intelligence with the next-token prediction paradigm in neural language modeling, which pointing out a free-lunch way to enhance the model performance by scaling the model parameters. In RecSys, the retrieval stage is also follows a 'next-token prediction' paradigm, to recall the hunderds of items from the global item set, thus the generative recommendation usually refers specifically to the retrieval stage (without Tree-based methods). This raises a philosophical question: without a ground-truth next item, does the generative recommendation also holds a potential scaling law? In retrospect, the generative recommendation has two different technique paradigms: (1) ANN-based framework, utilizing the compressed user embedding to retrieve nearest other items in embedding space, e.g, Kuaiformer. (2) Auto-regressive-based framework, employing the beam search to decode the item from whole space, e.g, OneRec. In this paper, we devise a unified encoder-decoder framework to validate their scaling-laws at same time. Our empirical finding is that both of their losses strictly adhere to power-law Scaling Laws ($R^2$>0.9) within our unified architecture.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。