对比两种推荐模型,发现直接用大模型做推荐更易扩展且效果更好。
Understanding Generative Recommendation with Semantic IDs from a Model-scaling View

- 用大模型直接生成推荐,比用语义编码表示物品更利于模型扩展。
- 在44M到14B参数规模下,大模型推荐比语义编码推荐性能提升最高达20%。
- 大模型能随规模增大更好捕捉用户行为数据,打破传统认知局限。
近年来,生成式推荐(GR)通过融合物品语义与协同过滤信号成为新范式。其中一种主流方法是使用从多模态编码器(如大语言或视觉模型)嵌入量化的语义ID(SIDs)来表示物品,并在自回归序列建模框架中进行推荐。然而,本文发现:当模型规模扩大时,基于SIDs的推荐系统性能迅速饱和——无论是扩大模态编码器、量化分词器还是推荐模型本身,均出现瓶颈。我们指出,根本原因在于SIDs对物品语义信息的表达能力有限。为此,我们重新考察另一种范式:直接以大语言模型(LLM)作为推荐器(LLM-as-RS)。实验表明,该范式具备更优的模型扩展性,在44M至14B参数范围内,其性能较最优的SID-based GR高出最多20%。同时,我们挑战了‘大模型难以捕捉协同信号’的普遍观点,发现随着模型规模增长,其建模用户-物品交互的能力反而增强。分析揭示了基于SIDs的推荐存在内在扩展极限,而将大模型作为推荐核心是迈向生成式推荐基础模型的重要路径。
原文摘要 · Abstract (English)
Recent advancements in generative models have allowed the emergence of a promising paradigm for recommender systems (RS), known as Generative Recommendation (GR), which tries to unify rich item semantics and collaborative filtering signals. One popular modern approach is to use semantic IDs (SIDs), which are discrete codes quantized from the embeddings of modality encoders (e.g., large language or vision models), to represent items in an autoregressive user interaction sequence modeling setup (henceforth, SID-based GR). While generative models in other domains exhibit well-established scaling laws, our work reveals that SID-based GR shows significant bottlenecks while scaling up the model. In particular, the performance of SID-based GR quickly saturates as we enlarge each component: the modality encoder, the quantization tokenizer, and the RS itself. In this work, we identify the limited capacity of SIDs to encode item semantic information as one of the fundamental bottlenecks. Motivated by this observation, as an initial effort to obtain GR models with better scaling behaviors, we revisit another GR paradigm that directly uses large language models (LLMs) as recommenders (henceforth, LLM-as-RS). Our experiments show that the LLM-as-RS paradigm has superior model scaling properties and achieves up to 20 percent improvement over the best achievable performance of SID-based GR through scaling. We also challenge the prevailing belief that LLMs struggle to capture collaborative filtering information, showing that their ability to model user-item interactions improves as LLMs scale up. Our analyses on both SID-based GR and LLMs across model sizes from 44M to 14B parameters underscore the intrinsic scaling limits of SID-based GR and position LLM-as-RS as a promising path toward foundation models for GR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。