将时间感知机制融入Mamba2架构,提升推荐系统效率与精度。
TiM4Rec: An Efficient Sequential Recommendation Model Based on Time-Aware Structured State Space Duality Model
- 设计新型时序结构掩码,高效融合时间感知能力到SSD框架
- 在低维场景下显著缓解性能下降,保持计算高效性
- 首个专为Mamba架构优化的时序推荐模型,适合高时效性场景
序列推荐正从Transformer转向Mamba架构,分为两代:基于状态空间模型(SSM)的Mamba1和基于状态空间对偶(SSD)的Mamba2。尽管SSD相比SSM具有更高计算效率,但在序列推荐任务中,尤其在关键的低维场景下表现退化。现有研究常采用时间感知方法缓解此问题,我们分析发现,通过借鉴时间感知机制可根本性补偿SSD的性能损失。因此,我们提出将时间感知融入SSD框架。然而,现有基于Transformer的时间感知方法(如TiSASRec)与SSD架构集成存在两大挑战:1)架构复杂度高;2)时间差建模需维度扩展,导致计算效率下降。为此,我们引入一种新型时间感知结构掩码,高效实现时间感知能力嵌入。基于此,提出面向推荐的时间感知Mamba(TiM4Rec),在低维场景下有效缓解性能下降,同时维持计算高效性。这是首个专为Mamba架构设计的时间感知增强方法,应用于序列推荐领域。在三个真实数据集上的大量实验验证了该方法的优越性。代码已公开于https://github.com/AlwaysFHao/TiM4Rec。
原文摘要 · Abstract (English)
The Sequential Recommendation modeling paradigm is shifting from Transformer to Mamba architecture, which comprises two generations: Mamba1, based on the State Space Model (SSM), and Mamba2, based on State Space Duality (SSD). Although SSD offers superior computational efficiency compared to SSM, it suffers performance degradation in sequential recommendation tasks, especially in low-dimensional scenarios that are critical for these tasks. Considering that time-aware enhancement methods are commonly employed to mitigate performance loss, our analysis reveals that the performance decline of SSD can similarly be fundamentally compensated by leveraging mechanisms in time-aware methods. Thus, we propose integrating time-awareness into the SSD framework to address these performance issues. However, integrating current time-aware methods, modeled after TiSASRec, into SSD faces the following challenges: 1) the complexity of integrating these transformer-based mechanisms with the SSD architecture, and 2) the computational inefficiency caused by the need for dimensionality expansion of time-difference modeling. To overcome these challenges, we introduce a novel Time-aware Structured Masked Matrix that efficiently incorporates time-aware capabilities into SSD. Building on this, we propose Time-Aware Mamba for Recommendation (TiM4Rec), which mitigates performance degradation in low-dimensional SSD contexts while preserving computational efficiency. This marks the inaugural application of a time-aware enhancement method specifically tailored for the Mamba architecture within the domain of sequential recommendation. Extensive experiments conducted on three real-world datasets demonstrate the superiority of our approach. The code for our model is accessible at https://github.com/AlwaysFHao/TiM4Rec.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。