解决扩散模型推荐中的嵌入坍缩问题，提升序列推荐准确率
Unlocking the Power of Diffusion Models in Sequential Recommendation: A Simple and Effective Approach
- 对每个物品独立加噪，在整个序列上进行扩散训练
- 三阶段训练策略有效缓解嵌入坍缩，精度显著提升
- 适合追求高精度序列推荐的工业与研究场景
本文针对现有基于扩散的序列推荐模型中常被忽视的嵌入坍缩问题，提出ADRec框架以缓解该问题。不同于以往方法，ADRec对每个序列项施加独立噪声，并在训练时对完整目标序列进行扩散。通过自回归捕捉项间依赖关系，同时利用项级扩散建模各项目分布。该双机制使模型能有效捕捉序列动态与物品表征，克服现有方法局限。为进一步缓解嵌入坍缩，提出三阶段训练策略：(1) 预训练嵌入权重，(2) 将其与ADRec主干对齐，(3) 微调模型。推理时仅对最后一个项执行去噪，确保历史交互中的有效模式得以保留。在六个数据集上的全面实验验证了ADRec在提升扩散型序列推荐系统的准确率与效率方面的有效性。
原文摘要 · Abstract (English)
In this paper, we focus on the often-overlooked issue of embedding collapse in existing diffusion-based sequential recommendation models and propose ADRec, an innovative framework designed to mitigate this problem. Diverging from previous diffusion-based methods, ADRec applies an independent noise process to each token and performs diffusion across the entire target sequence during training. ADRec captures token interdependency through auto-regression while modeling per-token distributions through token-level diffusion. This dual approach enables the model to effectively capture both sequence dynamics and item representations, overcoming the limitations of existing methods. To further mitigate embedding collapse, we propose a three-stage training strategy: (1) pre-training the embedding weights, (2) aligning these weights with the ADRec backbone, and (3) fine-tuning the model. During inference, ADRec applies the denoising process only to the last token, ensuring that the meaningful patterns in historical interactions are preserved. Our comprehensive empirical evaluation across six datasets underscores the effectiveness of ADRec in enhancing both the accuracy and efficiency of diffusion-based sequential recommendation systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。