arXiv:2409.17165cs.IRcs.LG2024-09被引 1

用Mamba替代Transformer,提升推荐系统效率与可扩展性。

Mamba for Scalable and Efficient Personalized Recommendations

  • 将Mamba层融入FT-Transformer,以线性复杂度替代传统Transformer的二次复杂度。
  • 在三个数据集上训练16万用户行为对,精度、召回率等指标均优于或持平于Transformer模型。
  • 适合需要高效处理大规模用户行为数据的推荐系统研发人员参考。

本文提出使用Mamba处理个性化推荐系统中的表格数据。我们设计了 extit{FT-Mamba}(Feature Tokenizer + Mamba)这一新型混合模型,将FT-Transformer架构中的Transformer层替换为Mamba层,用于处理推荐系统中的表格数据。Mamba模型通过增强状态空间模型(SSMs)的能力,将计算复杂度从二次降低至线性,提供比Transformer更高效的替代方案。FT-Mamba旨在提升推荐系统的可扩展性与效率,同时保持性能。我们在三个数据集上进行了评估:Spotify音乐推荐、H&M时尚推荐和疫苗信息推荐。每个模型均在16万条用户行为对上训练,使用精确率(P)、召回率(R)、平均倒数排名(MRR)和命中率(HR)在多个截断值下衡量性能。结果表明,相较于基于Transformer的模型,FT-Mamba在计算效率上表现更优,且在关键推荐指标上维持或超越其性能。通过引入Mamba层,FT-Mamba为大规模个性化推荐系统提供了高效可靠的解决方案,展现了Mamba架构在提升效率与准确率方面的潜力。

原文摘要 · Abstract (English)

In this effort, we propose using the Mamba for handling tabular data in personalized recommendation systems. We present the \textit{FT-Mamba} (Feature Tokenizer\,$+$\,Mamba), a novel hybrid model that replaces Transformer layers with Mamba layers within the FT-Transformer architecture, for handling tabular data in personalized recommendation systems. The \textit{Mamba model} offers an efficient alternative to Transformers, reducing computational complexity from quadratic to linear by enhancing the capabilities of State Space Models (SSMs). FT-Mamba is designed to improve the scalability and efficiency of recommendation systems while maintaining performance. We evaluate FT-Mamba in comparison to a traditional Transformer-based model within a Two-Tower architecture on three datasets: Spotify music recommendation, H\&M fashion recommendation, and vaccine messaging recommendation. Each model is trained on 160,000 user-action pairs, and performance is measured using precision (P), recall (R), Mean Reciprocal Rank (MRR), and Hit Ratio (HR) at several truncation values. Our results demonstrate that FT-Mamba outperforms the Transformer-based model in terms of computational efficiency while maintaining or exceeding performance across key recommendation metrics. By leveraging Mamba layers, FT-Mamba provides a scalable and effective solution for large-scale personalized recommendation systems, showcasing the potential of the Mamba architecture to enhance both efficiency and accuracy.

推荐系统Mamba高效建模表格数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。