arXiv:2508.02050cs.IR2025-08中稿 · ACMMM 2025被引 1

用生成式注意力替代传统机制,更好捕捉用户兴趣变化。

Why Generate When You Can Transform? Unleashing Generative Attention for Dynamic Recommendation

  • 通过变分自编码器和扩散模型生成动态注意力分布
  • 在真实数据集上准确率与多样性显著超越现有方法
  • 适合关注用户行为建模与推荐系统创新的研究者

序列推荐(SR)通过分析用户历史行为预测未来偏好,以实现个性化体验。尽管基于自注意力机制的Transformer已成为主流架构,但传统注意力计算方式本质上是线性且确定性的,难以捕捉用户偏好的动态演化与复杂模式。鉴于生成模型擅长建模非线性与概率性特征,本文提出生成式注意力机制更具表达力与随机性。理论证明表明,该方法在表达能力上优于传统确定性方法。基于此,我们构建了两种分别基于变分自编码器(VAE)与扩散模型(DM)的生成式注意力模型,用于生成适配用户动态偏好的注意力分布。在多个真实数据集上的实验表明,所提模型在推荐准确率与多样性方面均显著优于当前最先进方法。

原文摘要 · Abstract (English)

Sequential Recommendation (SR) focuses on personalizing user experiences by predicting future preferences based on historical interactions. Transformer models, with their attention mechanisms, have become the dominant architecture in SR tasks due to their ability to capture dependencies in user behavior sequences. However, traditional attention mechanisms, where attention weights are computed through query-key transformations, are inherently linear and deterministic. This fixed approach limits their ability to account for the dynamic and non-linear nature of user preferences, leading to challenges in capturing evolving interests and subtle behavioral patterns. Given that generative models excel at capturing non-linearity and probabilistic variability, we argue that generating attention distributions offers a more flexible and expressive alternative compared to traditional attention mechanisms. To support this claim, we present a theoretical proof demonstrating that generative attention mechanisms offer greater expressiveness and stochasticity than traditional deterministic approaches. Building upon this theoretical foundation, we introduce two generative attention models for SR, each grounded in the principles of Variational Autoencoders (VAE) and Diffusion Models (DMs), respectively. These models are designed specifically to generate adaptive attention distributions that better align with variable user preferences. Extensive experiments on real-world datasets show our models significantly outperform state-of-the-art in both accuracy and diversity.

序列推荐生成式模型注意力机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。