将用户行为拆解为多个主题序列,提升推荐系统精准度
CMSL: Constructive Multi-Sequence Learning for Recommendation Systems

- 通过可学习模块在隐空间构建多条连贯行为序列
- 在Meta四大场景部署,显著降低信号干扰
- 适合关注推荐系统结构优化的研究者与工程师
序列学习已成为推荐系统中极具潜力的范式,超越传统深度学习推荐模型(DLRM),能够捕捉用户行为的时间动态。然而,当前最先进的架构仍受限于一个类比:将用户历史视为类似大语言模型(LLM)中句子的单一时间序列。我们观察到自然语言与推荐数据的根本差异:文本具有线性逻辑流,而用户行为是多维度的碎片化记录,不同兴趣间缺乏强连贯性,导致上下文污染。在单序列建模中,无关行为争夺相同的注意力资源,产生“噪声”信号,削弱模型对高意图模式的识别能力。为此,我们提出构造性多序列学习(CMSL),实现从被动序列接收向主动“上下文工程”的范式转变,即在隐空间中构建多个连贯的主题序列。CMSL利用可学习的序列构建模块将用户历史解耦为“纯净”的主题轨迹,并通过线性注意力机制高效建模这些轨迹。CMSL已在Meta的排名与检索任务中部署,覆盖四个主要产品表面。
原文摘要 · Abstract (English)
Sequence learning has emerged as the promising paradigm in recommendation systems, surpassing traditional Deep Learning Recommendation Models (DLRM) by capturing the temporal nuances of user behavior. However, current state-of-the-art architectures operate under a limiting analogy: they treat user history as a monolithic chronological sequence like a sentence in a Large Language Model (LLM). We observe a fundamental divergence between natural language and recommendation data: unlike the linear, logical flow of text, user history is inherently multi-faceted. A user's journey is a fragmented reflection of diverse interests, resulting in much weaker coherence between items than is found in LLM training data. This lack of structural unity leads to context pollution. In single-sequence modeling, unrelated behaviors compete for the same attention budget. This "noisy" signal dilutes the model's focus, effectively capping its ability to discern high-intent patterns from background activity. To address this, we propose Constructive Multi-Sequence Learning (CMSL), a paradigm shift from passive sequence ingestion to active "context engineering" that constructs multiple coherent sequences in latent space. CMSL leverages a learnable Sequence Construction Module to disentangle user history into "pure" thematic strands, followed by a linear attention mechanism to efficiently model these strands at scale. CMSL has been deployed across ranking and retrieval tasks and across four major surfaces at Meta.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。