用扩散模型替代自回归生成,提升推荐系统效率与效果
Masked Diffusion Generative Recommendation
- 用并行码本和掩码机制重构推荐生成流程
- 在多个数据集上最高提升10.78%性能,线上平台收入增1.20%
- 适合追求高效实时推荐的工业级应用
生成式推荐(GR)通常先将连续物品嵌入量化为多级语义ID(SIDs),再通过自回归解码生成下一个物品。尽管现有方法表现良好,但直接沿用语言模型的自回归范式仍存在三大局限:(1) 自回归解码难以联合捕捉不同位置SID间多维特征的全局依赖;(2) 对同一物品使用统一固定解码路径,隐含假设所有用户以相同顺序关注物品属性;(3) 推理效率低,难以满足实时需求。为此,我们提出MDGR——一种基于扩散模型的生成式推荐框架,从码本、训练和推理三方面重构GR流程:(1) 采用并行码本为扩散生成提供结构基础;(2) 训练时在时间和样本维度自适应构建掩码监督信号;(3) 推理时设计基于预热的两阶段并行解码策略,实现SIDs高效生成。大量实验表明,MDGR在多个公开及工业级数据集上超越十种先进基线,最高提升10.78%。此外,在大规模在线广告平台部署后,实现1.20%的收入增长,验证其实际价值。
原文摘要 · Abstract (English)
Generative recommendation (GR) typically first quantizes continuous item embeddings into multi-level semantic IDs (SIDs), and then generates the next item via autoregressive decoding. Although existing methods are already competitive in terms of recommendation performance, directly inheriting the autoregressive decoding paradigm from language models still suffers from three key limitations: (1) autoregressive decoding struggles to jointly capture global dependencies among the multi-dimensional features associated with different positions of SID; (2) using a unified, fixed decoding path for the same item implicitly assumes that all users attend to item attributes in the same order; (3) autoregressive decoding is inefficient at inference time and struggles to meet real-time requirements. To tackle these challenges, we propose MDGR, a Masked Diffusion Generative Recommendation framework that reshapes the GR pipeline from three perspectives: codebook, training, and inference. (1) We adopt a parallel codebook to provide a structural foundation for diffusion-based GR. (2) During training, we adaptively construct masking supervision signals along both the temporal and sample dimensions. (3) During inference, we develop a warm-up-based two-stage parallel decoding strategy for efficient generation of SIDs. Extensive experiments on multiple public and industrial-scale datasets show that MDGR outperforms ten state-of-the-art baselines by up to 10.78%. Furthermore, by deploying MDGR on a large-scale online advertising platform, we achieve a 1.20% increase in revenue, demonstrating its practical value.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。