用文本预测提升推荐模型,减少对海量数据依赖。
JEPA4Rec: Learning Effective Language Representations for Sequential Recommendation via Joint Embedding Predictive Architecture
- 将商品描述转为句子,通过掩码预测学习通用物品表示
- 在6个真实数据集上优于现有方法,低资源场景表现更优
- 适合做跨域、跨平台推荐,尤其数据稀疏时效果显著
语言表示学习为序列推荐提供了有前景的解决方案,因其能学习可迁移的表示。然而,该方法仍面临数据稀疏和对常识性用户偏好理解不足的问题。为此,我们提出JEPA4Rec,结合联合嵌入预测架构与物品文本描述的语言建模。JEPA4Rec通过整合标题、类别等属性,将物品表示为文本句子,并采用改进的双向Transformer编码器进行编码。对句子实施掩码后,模型预测未掩码部分的表示,从而学习通用物品嵌入。为进一步提升性能,采用两阶段自监督训练策略。在六个真实世界数据集上的实验表明,JEPA4Rec持续优于当前最优方法,尤其在跨域、跨平台及低资源场景中表现突出。
原文摘要 · Abstract (English)
Language representation learning has emerged as a promising approach for sequential recommendation, thanks to its ability to learn generalizable representations. However, despite its advantages, this approach still struggles with data sparsity and a limited understanding of common-sense user preferences. To address these limitations, we propose $\textbf{JEPA4Rec}$, a framework that combines $\textbf{J}$oint $\textbf{E}$mbedding $\textbf{P}$redictive $\textbf{A}$rchitecture with language modeling of item textual descriptions. JEPA4Rec captures semantically rich and transferable representations, improving recommendation performance and reducing reliance on large-scale pre-training data. Specifically, JEPA4Rec represents items as text sentences by flattening descriptive information such as $\textit{title, category}$, and other attributes. To encode these sentences, we employ a bidirectional Transformer encoder with modified embedding layers tailored for capturing item information in recommendation datasets. We apply masking to text sentences and use them to predict the representations of the unmasked sentences, helping the model learn generalizable item embeddings. To further improve recommendation performance and language understanding, we employ a two-stage training strategy incorporating self-supervised learning losses. Experiments on six real-world datasets demonstrate that JEPA4Rec consistently outperforms state-of-the-art methods, particularly in cross-domain, cross-platform, and low-resource scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。