用扩散模型生成推荐序列,提速并提升效果。
Masked Diffusion for Generative Recommendation
- 用掩码扩散建模用户序列分布,支持并行解码。
- 在数据少时表现更优,粗粒度召回率显著提升。
- 适合需要高效推理的推荐系统场景。
基于语义ID(SID)的生成式推荐(GR)因其性能提升、利用语言模型嵌入的语义信息以及推理与存储效率,成为传统推荐方法的有前景替代方案。现有基于SID的GR方法采用自回归建模用户交互序列的概率。尽管在某些场景下实现了优异的下一项目预测性能,但这类方法因逐词顺序解码而计算成本高,且训练数据利用不充分,易偏向学习短上下文关系。受自然语言处理领域最新突破启发,我们提出改用掩码扩散模型来建模和学习用户序列的分布。该方法通过离散掩码噪声促进序列分布学习,并假设被掩码项在未掩码项条件下条件独立,从而实现被掩码项的并行解码。大量实验表明,所提方法在各项指标上均持续优于自回归建模,尤其在数据受限场景和粗粒度召回方面差距明显。此外,该方法可在推理时并行预测多个SID,同时保持对自回归方法的性能优势。
原文摘要 · Abstract (English)
Generative recommendation (GR) with semantic IDs (SIDs) has emerged as a promising alternative to traditional recommendation approaches due to its performance gains, capitalization on semantic information provided through language model embeddings, and inference and storage efficiency. Existing GR with SIDs works frame the probability of a sequence of SIDs corresponding to a user's interaction history using autoregressive modeling. While this has led to impressive next item prediction performances in certain settings, these autoregressive GR with SIDs models suffer from expensive inference due to sequential token-wise decoding, potentially inefficient use of training data and bias towards learning short-context relationships among tokens. Inspired by recent breakthroughs in NLP, we propose to instead model and learn the probability of a user's sequence of SIDs using masked diffusion. Masked diffusion employs discrete masking noise to facilitate learning the sequence distribution, and models the probability of masked tokens as conditionally independent given the unmasked tokens, allowing for parallel decoding of the masked tokens. We demonstrate through thorough experiments that our proposed method consistently outperforms autoregressive modeling. This performance gap is especially pronounced in data-constrained settings and in terms of coarse-grained recall, consistent with our intuitions. Moreover, our approach allows the flexibility of predicting multiple SIDs in parallel during inference while maintaining superior performance to autoregressive modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。