提出BlossomRec,用分块稀疏注意力同时捕捉用户长短期兴趣。
BlossomRec: Block-level Fused Sparse Attention Mechanism for Sequential Recommendations
- 分块设计两种稀疏注意力,分别处理长期与短期兴趣
- 在4个数据集上显著降低内存占用,性能持平或更优
- 适合需要高效建模长序列的推荐系统场景
Transformer结构广泛应用于序列推荐系统(SRS),但随着用户交互历史增长,计算时间和内存需求也随之上升,主要源于标准注意力机制。尽管已有诸多高效注意力和SSM-based模型,这些方法在长序列建模上表现不佳,且对短序列可能产生不稳定性能。为此,我们设计了一种名为BlossomRec的稀疏注意力机制,通过分块计算不同模式的注意力来同时建模用户的长期与短期兴趣,实现跨不同长度序列的稳定性能。具体而言,将用户兴趣分为长期与短期两类,采用两种不同的稀疏注意力模式分别计算,并通过可学习门控输出融合结果。理论上,该方法显著减少了参与注意力计算的交互次数。在四个公开数据集上的大量实验表明,当与当前最先进的基于Transformer的模型结合时,BlossomRec在保持甚至超越性能的同时,显著降低了内存使用量,充分证明了其高效性与有效性。代码已开源:https://github.com/Applied-Machine-Learning-Lab/WWW2026_BlossomRec。
原文摘要 · Abstract (English)
Transformer structures have been widely used in sequential recommender systems (SRS). However, as user interaction histories increase, computational time and memory requirements also grow. This is mainly caused by the standard attention mechanism. Although there exist many methods employing efficient attention and SSM-based models, these approaches struggle to effectively model long sequences and may exhibit unstable performance on short sequences. To address these challenges, we design a sparse attention mechanism, BlossomRec, which models both long-term and short-term user interests through attention computation to achieve stable performance across sequences of varying lengths. Specifically, we categorize user interests in recommendation systems into long-term and short-term interests, and compute them using two distinct sparse attention patterns, with the results combined through a learnable gated output. Theoretically, it significantly reduces the number of interactions participating in attention computation. Extensive experiments on four public datasets demonstrate that BlossomRec, when integrated with state-of-the-art Transformer-based models, achieves comparable or even superior performance while significantly reducing memory usage, providing strong evidence of BlossomRec's efficiency and effectiveness. The code is available at https://github.com/Applied-Machine-Learning-Lab/WWW2026_BlossomRec.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。