arXiv:2502.10833cs.IR2025-02被引 54

提出无序标识符让大模型推荐更准更快,支持同时生成。

Order-agnostic Identifier for Large Language Model-based Generative Recommendation

  • 用无序集合标识符融合协同过滤与语义信息
  • 支持一次性生成,效率提升且避免局部最优
  • 适合冷启动和大规模推荐场景,模型越大越有效

基于大语言模型的生成式推荐研究日益活跃,其中物品标识符设计至关重要。现有方法使用序列化或单标记符,前者存在束搜索局部最优和逐步生成效率低的问题,后者难以捕捉丰富语义或协同过滤信息。为此,本文提出两个基本原则:融合协同过滤与语义信息以捕获多维物品特征,设计无序标识符以消除标记依赖性。据此,提出SETRec新范式,将每个物品表示为一组无序标记。通过协同过滤与语义分词器获取多维度无序标记,采用稀疏注意力掩码编码用户历史,并利用查询引导生成机制实现并行生成。在T5和Qwen(1.5B至7B)上验证,SETRec在全排名、热启与冷启推荐及不同物品流行度组中均表现优异,且生成效率显著提升,在模型规模扩大时对冷启动物品仍具良好可扩展性。

原文摘要 · Abstract (English)

Leveraging Large Language Models (LLMs) for generative recommendation has attracted significant research interest, where item tokenization is a critical step. It involves assigning item identifiers for LLMs to encode user history and generate the next item. Existing approaches leverage either token-sequence identifiers, representing items as discrete token sequences, or single-token identifiers, using ID or semantic embeddings. Token-sequence identifiers face issues such as the local optima problem in beam search and low generation efficiency due to step-by-step generation. In contrast, single-token identifiers fail to capture rich semantics or encode Collaborative Filtering (CF) information, resulting in suboptimal performance. To address these issues, we propose two fundamental principles for item identifier design: 1) integrating both CF and semantic information to fully capture multi-dimensional item information, and 2) designing order-agnostic identifiers without token dependency, mitigating the local optima issue and achieving simultaneous generation for generation efficiency. Accordingly, we introduce a novel set identifier paradigm for LLM-based generative recommendation, representing each item as a set of order-agnostic tokens. To implement this paradigm, we propose SETRec, which leverages CF and semantic tokenizers to obtain order-agnostic multi-dimensional tokens. To eliminate token dependency, SETRec uses a sparse attention mask for user history encoding and a query-guided generation mechanism for simultaneous token generation. We instantiate SETRec on T5 and Qwen (from 1.5B to 7B). Extensive experiments demonstrate its effectiveness under various scenarios (e.g., full ranking, warm- and cold-start ranking, and various item popularity groups). Moreover, results validate SETRec's superior efficiency and show promising scalability on cold-start items as model sizes increase.

生成推荐大模型无序标识冷启动

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。