arXiv:2601.12301cs.IR2026-01

让推荐系统理解物品的多个维度特征,提升个性化推荐效果

Facet-Aware Multi-Head Mixture-of-Experts Model with Text-Enhanced Pre-training for Sequential Recommendation

  • 用多头注意力提取物品不同特征面,动态加权融合预测结果
  • 引入专家混合网络捕捉用户在各特征面内的复杂偏好变化
  • 通过文本预训练增强物品嵌入,使特征表达更语义化

序列推荐系统通过用户交互历史捕捉动态偏好。现有方法通常为每个物品分配单一嵌入向量,难以充分表征物品的多维属性(如电影类型、主演)。用户在同一维度内也可能有复杂偏好(如同时喜欢动作片和音乐片)。为此,我们提出面向特征感知的多头专家混合模型(FAME)。利用多头注意力最后一层各头的子嵌入分别预测下一个物品,有效捕捉物品的不同特征面;通过门控机制动态决定各预测的重要性。此外,在每个注意力头中引入专家混合(MoE)网络,通过可学习路由网络根据上下文聚合专家输出,以解耦各特征面内的用户偏好。同时设计文本增强型特征感知预训练模块:利用预训练文本编码器与交替监督对比学习目标,从文本元数据(如描述)中显式解耦出特征特定信息,确保物品嵌入在序列训练前即具备语义鲁棒性,与多维度框架对齐。

原文摘要 · Abstract (English)

Sequential recommendation (SR) systems excel at capturing users' dynamic preferences by leveraging their interaction histories. Most existing SR systems assign a single embedding vector to each item to represent its features, adopting various models to combine these embeddings into a sequence representation that captures user intent. However, we argue that this representation alone is insufficient to capture an item's multi-faceted nature (e.g., movie genres, starring actors). Furthermore, users often exhibit complex and varied preferences within these facets (e.g., liking both action and musical films within the genre facet), which are challenging to fully represent with static identifiers. To address these issues, we propose a novel architecture titled Facet-Aware Multi-Head Mixture-of-Experts Model for Sequential Recommendation (FAME). We leverage sub-embeddings from each head in the final multi-head attention layer to predict the next item separately, effectively capturing distinct item facets. A gating mechanism then integrates these predictions by dynamically determining their importance. Additionally, we introduce a Mixture-of-Experts (MoE) network within each attention head to disentangle varied user preferences within each facet, utilizing a learnable router network to aggregate expert outputs based on context. Complementing this architecture, we design a Text-Enhanced Facet-Aware Pre-training module to overcome the limitations of randomly initialized embeddings. By utilizing a pre-trained text encoder and employing an alternating supervised contrastive learning objective, we explicitly disentangle facet-specific features from textual metadata (e.g., descriptions) before sequential training begins. This ensures that the item embeddings are semantically robust and aligned with the downstream multi-facet framework.

序列推荐多面特征MoE文本预训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。