提出新方法提升纯编码器Transformer在单会话推荐中的表现
Optimizing Encoder-Only Transformers for Session-Based Recommendation Systems
- 用滑动窗口增强数据，结合末尾标记掩码捕捉序列依赖
- 在Yoochoose、Diginetica、Tmall上均超越同类模型
- 适合研究会话推荐与Transformer结构优化的学者
会话推荐旨在预测用户下一个交互项目，通常无法获取历史用户数据。本文针对纯编码器Transformer架构，提出序列掩码建模（Sequential Masked Modeling）方法，通过滑动窗口数据增强与独特的末尾标记掩码策略，更有效地捕捉序列依赖关系。该方法显著提升了下一项目预测性能。我们在Yoochoose 1/64、Diginetica、Tmall三个常用数据集上评估，对比了当前最优的单会话、跨会话及多关系方法。结果表明，所提出的Transformer-SMM模型在仅使用相同信息量的前提下持续领先，甚至媲美利用更多用户历史信息的方法。本研究凸显了纯编码器Transformer在会话推荐中的潜力，并为后续改进开辟路径。
原文摘要 · Abstract (English)
Session-based recommendation is the task of predicting the next item a user will interact with, often without access to historical user data. In this work, we introduce Sequential Masked Modeling, a novel approach for encoder-only transformer architectures to tackle the challenges of single-session recommendation. Our method combines data augmentation through window sliding with a unique penultimate token masking strategy to capture sequential dependencies more effectively. By enhancing how transformers handle session data, Sequential Masked Modeling significantly improves next-item prediction performance. We evaluate our approach on three widely-used datasets, Yoochoose 1/64, Diginetica, and Tmall, comparing it to state-of-the-art single-session, cross-session, and multi-relation approaches. The results demonstrate that our Transformer-SMM models consistently outperform all models that rely on the same amount of information, while even rivaling methods that have access to more extensive user history. This study highlights the potential of encoder-only transformers in session-based recommendation and opens the door for further improvements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。