arXiv:2512.11254cs.IR2025-12

聚焦注意力提升生成式推荐,让模型更关注关键信息。

FAIR: Focused Attention Is All You Need for Generative Recommendation

  • 引入双注意力权重差分机制,过滤无关噪声,聚焦有效上下文。
  • 在四个公开数据集上显著优于现有方法,性能提升明显。
  • 适合处理含噪用户行为序列的推荐系统研究者使用。

近期基于Transformer的生成式推荐在用户行为建模中受到广泛关注。然而,其通常需将物品离散化为多码表示(如通常至少四个码令牌),大幅增加原始物品序列长度。这种扩展使基于Transformer的模型在建模具有内在噪声的用户行为序列时面临挑战,因其容易过度关注无关或噪声上下文。为此,我们提出FAIR,首个具备聚焦注意力的生成式推荐框架,通过增强对相关上下文的关注并抑制对无关内容的关注来缓解该问题。具体而言,我们提出:(1) 一种集成于标准Transformer的聚焦注意力机制,学习两组独立的Q和K注意力权重,并以它们的差值作为最终注意力分数,从而消除注意力噪声并聚焦于相关上下文;(2) 一种噪声鲁棒性目标,促使模型在随机扰动下保持稳定的注意力模式,防止因噪声导致注意力错误偏移至无关上下文;(3) 一种互信息最大化目标,引导模型识别对下一步物品预测最具信息量的上下文。我们在四个公开基准数据集上验证了FAIR的有效性,结果表明其性能显著优于现有方法。

原文摘要 · Abstract (English)

Recently, transformer-based generative recommendation has garnered significant attention for user behavior modeling. However, it often requires discretizing items into multi-code representations (e.g., typically four code tokens or more), which sharply increases the length of the original item sequence. This expansion poses challenges to transformer-based models for modeling user behavior sequences with inherent noises, since they tend to overallocate attention to irrelevant or noisy context. To mitigate this issue, we propose FAIR, the first generative recommendation framework with focused attention, which enhances attention scores to relevant context while suppressing those to irrelevant ones. Specifically, we propose (1) a focused attention mechanism integrated into the standard Transformer, which learns two separate sets of Q and K attention weights and computes their difference as the final attention scores to eliminate attention noise while focusing on relevant contexts; (2) a noise-robustness objective, which encourages the model to maintain stable attention patterns under stochastic perturbations, preventing undesirable shifts toward irrelevant context due to noise; and (3) a mutual information maximization objective, which guides the model to identify contexts that are most informative for next-item prediction. We validate the effectiveness of FAIR on four public benchmarks, demonstrating its superior performance compared to existing methods.

生成式推荐注意力机制噪声鲁棒

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。