让注意力关注自身权重,提升推荐系统对物品关系的捕捉能力
Pay Attention to Attention for Sequential Recommendation
- 通过二次关注注意力权重,优化物品间关联的分布
- 在多个真实数据集上优于当前最优推荐模型
- 适合需要挖掘复杂序列依赖的推荐场景
基于Transformer的方法在多种序列任务中表现卓越,但传统的自注意力模型在序列推荐场景中难以充分捕捉物品间的复杂依赖关系,原因在于注意力权重缺乏显式强调,而其在分配关注和理解物品关联中起关键作用。为此,我们提出一种新型序列推荐方法——注意力权重精炼(AWRSR),通过额外关注注意力权重,使物品间相关性的注意力分布更加精细。我们在多个真实数据集上进行了全面实验,结果表明该方法持续优于当前最先进的序列推荐模型。此外,我们还深入分析了AWRSR在捕捉高层级依赖方面的有效性。研究结果表明,AWRSR为提升自注意力架构在序列推荐任务中的性能提供了新方向,并可能推广至其他序列建模问题。
原文摘要 · Abstract (English)
Transformer-based approaches have demonstrated remarkable success in various sequence-based tasks. However, traditional self-attention models may not sufficiently capture the intricate dependencies within items in sequential recommendation scenarios. This is due to the lack of explicit emphasis on attention weights, which play a critical role in allocating attention and understanding item-to-item correlations. To better exploit the potential of attention weights and improve the capability of sequential recommendation in learning high-order dependencies, we propose a novel sequential recommendation (SR) approach called attention weight refinement (AWRSR). AWRSR enhances the effectiveness of self-attention by additionally paying attention to attention weights, allowing for more refined attention distributions of correlations among items. We conduct comprehensive experiments on multiple real-world datasets, demonstrating that our approach consistently outperforms state-of-the-art SR models. Moreover, we provide a thorough analysis of AWRSR's effectiveness in capturing higher-level dependencies. These findings suggest that AWRSR offers a promising new direction for enhancing the performance of self-attention architecture in SR tasks, with potential applications in other sequence-based problems as well.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。