arXiv:2511.06254cs.IRcs.CL2025-11被引 12

用扩散模型并行生成推荐项的语义编码,解决传统方法依赖顺序预测的问题。

LLaDA-Rec: Discrete Diffusion for Parallel Semantic ID Generation in Generative Recommendation

  • 采用双向注意力与自适应生成顺序,实现并行语义编码。
  • 在三个真实数据集上超越现有生成推荐方法,效果提升显著。
  • 适合关注生成式推荐新范式的研究者和工程师。

生成式推荐将每个物品表示为语义ID(离散标记序列),通过自回归解码生成下一个物品。现有自回归模型存在两大固有缺陷:(1) 单向约束,因果注意力限制每个标记仅能关注其前序标记,阻碍全局语义建模;(2) 错误累积,固定左右顺序导致早期标记预测误差传播至后续标记。为此,我们提出LLaDA-Rec,一种离散扩散框架,将推荐重构为并行语义ID生成。结合双向注意力与自适应生成顺序,该方法更有效地建模项间与项内依赖关系,并缓解错误累积。具体包括三项设计:(1) 并行分词方案,生成支持双向建模的语义ID,解决残差量化与双向架构间的不匹配;(2) 用户历史与目标物品层面的双重掩码机制,捕捉项间序列依赖与项内语义关系;(3) 适配离散扩散解码的改进束搜索策略,克服标准束搜索与扩散生成的不兼容性。在三个真实世界数据集上的实验表明,LLaDA-Rec持续优于基于ID及当前最先进生成推荐模型,确立离散扩散作为生成推荐新范式。

原文摘要 · Abstract (English)

Generative recommendation represents each item as a semantic ID, i.e., a sequence of discrete tokens, and generates the next item through autoregressive decoding. While effective, existing autoregressive models face two intrinsic limitations: (1) unidirectional constraints, where causal attention restricts each token to attend only to its predecessors, hindering global semantic modeling; and (2) error accumulation, where the fixed left-to-right generation order causes prediction errors in early tokens to propagate to the predictions of subsequent token. To address these issues, we propose LLaDA-Rec, a discrete diffusion framework that reformulates recommendation as parallel semantic ID generation. By combining bidirectional attention with the adaptive generation order, the approach models inter-item and intra-item dependencies more effectively and alleviates error accumulation. Specifically, our approach comprises three key designs: (1) a parallel tokenization scheme that produces semantic IDs for bidirectional modeling, addressing the mismatch between residual quantization and bidirectional architectures; (2) two masking mechanisms at the user-history and next-item levels to capture both inter-item sequential dependencies and intra-item semantic relationships; and (3) an adapted beam search strategy for adaptive-order discrete diffusion decoding, resolving the incompatibility of standard beam search with diffusion-based generation. Experiments on three real-world datasets show that LLaDA-Rec consistently outperforms both ID-based and state-of-the-art generative recommenders, establishing discrete diffusion as a new paradigm for generative recommendation.

生成推荐离散扩散并行生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。