用多嵌入与专家混合模型提升推荐系统可扩展性
Revisiting scalable sequential recommendation with Multi-Embedding Approach and Mixture-of-Experts
- 将单个嵌入矩阵拆分为多个低维矩阵，解耦捕捉物品多维特征
- 引入专家混合层实现用户上下文下特征的自适应变换
- 在公开数据集上优于多个基线模型，适合大规模推荐场景
在推荐系统中，如何有效扩展推荐模型一直是核心研究课题。尽管顺序推荐（SR）模型已发展出先进且可扩展的架构，但仍面临物品多维度特征及用户上下文中动态相关性的挑战。为此，我们提出Fuxi-MME框架，融合多嵌入策略与专家混合（MoE）架构。具体地，为高效解耦地捕捉多样化的物品特征，我们将传统的单个嵌入矩阵分解为多个低维嵌入矩阵。此外，通过用MoE层替代Fuxi Block中的相关参数，模型实现了对丰富表示的自适应、专业化转换。在公开数据集上的实验结果表明，所提框架优于多个竞争性基线。
原文摘要 · Abstract (English)
In recommendation systems, how to effectively scale up recommendation models has been an essential research topic. While significant progress has been made in developing advanced and scalable architectures for sequential recommendation(SR) models, there are still challenges due to items' multi-faceted characteristics and dynamic item relevance in the user context. To address these issues, we propose Fuxi-MME, a framework that integrates a multi-embedding strategy with a Mixture-of-Experts (MoE) architecture. Specifically, to efficiently capture diverse item characteristics in a decoupled manner, we decompose the conventional single embedding matrix into several lower-dimensional embedding matrices. Additionally, by substituting relevant parameters in the Fuxi Block with an MoE layer, our model achieves adaptive and specialized transformation of the enriched representations. Empirical results on public datasets show that our proposed framework outperforms several competitive baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。