arXiv:2604.20858cs.IRcs.AI2026-04被引 7

通过主题感知路由和多尺度融合,提升长序列推荐的准确性与效率

Mixture of Sequence: Theme-Aware Mixture-of-Experts for Long-Sequence Recommendation

论文配图:Mixture of Sequence: Theme-Aware Mixture-of-Experts for Long-Sequence Recommendation
图 1 · 摘自论文原文
  • 基于主题感知路由将用户行为序列拆分为聚焦特定主题的子序列
  • 在多个数据集上达到当前最佳性能,且计算量低于其他MoE模型
  • 适合需要处理用户兴趣频繁切换的长序列推荐场景

序列推荐在点击率预测中快速发展,因其能建模动态用户兴趣。但长序列建模面临重大挑战:用户兴趣常发生显著转移,引入大量无关或误导信息。实证分析验证了这一问题,并发现长序列中存在重复出现的“会话跳跃”模式——用户兴趣在短时间窗口内稳定,跨会话则剧烈变化,可能经历多次会话后重现。为此,我们提出Mixture of Sequence(MoS)框架,一种模型无关的MoE方法,通过从原始用户序列中提取主题特异性和多尺度子序列实现精准预测。首先,MoS采用主题感知路由机制,自适应学习用户序列的潜在主题,并将序列组织为多个语义一致的子序列,每个子序列仅包含与特定主题匹配的会话,从而有效过滤因兴趣跳变带来的干扰信息。此外,为缓解信息损失,引入多尺度融合机制,利用三类专家捕捉全局序列特征、短期行为模式及主题特异性语义。二者协同使MoS能从多维度、多尺度视角提供准确推荐。实验表明,MoS在多个数据集上持续达到最先进性能,同时相比其他MoE模型引入更少的浮点运算量(FLOPs),充分证明其在性能与效率间的优异平衡。代码已开源。

原文摘要 · Abstract (English)

Sequential recommendation has rapidly advanced in click-through rate prediction due to its ability to model dynamic user interests. A key challenge, however, lies in modeling long sequences: users often exhibit significant interest shifts, introducing substantial irrelevant or misleading information. Our empirical analysis corroborates this challenge and uncovers a recurring behavioral pattern in long sequences (\textit{session hopping}): user interests remain stable within short temporal spans (\textit{sessions}) but shift drastically across sessions and may reappear after multiple sessions. To address this challenge, we propose the Mixture of Sequence (MoS) framework, a model-agnostic MoE approach that achieves accurate predictions by extracting theme-specific and multi-scale subsequences from noisy raw user sequences. First, MoS employs a theme-aware routing mechanism to adaptively learn the latent themes of user sequences and organizes these sequences into multiple coherent subsequences. Each subsequence contains only sessions aligned with a specific theme, thereby effectively filtering out irrelevant or even misleading information introduced by user interest shifts in session hopping. In addition, to alleviate potential information loss, we introduce a multi-scale fusion mechanism, which leverages three types of experts to capture global sequence characteristics, short-term user behaviors, and theme-specific semantic patterns. Together, these two mechanisms endow MoS with the ability to deliver accurate recommendations from multi-faceted and multi-scale perspectives. Experimental results demonstrate that MoS consistently achieves the SOTA performance while introducing fewer FLOPs compared with other MoE counterparts, providing strong evidence of its excellent balance between utility and efficiency. The code is available at https://github.com/xiaolin-cs/MoS.

序列推荐MoE主题建模长序列

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。