用滑动窗口训练让推荐模型学得更久远的用户偏好。
Sliding Window Training -- Utilizing Historical Recommender Systems Data for Foundation Models
- 训练时用滑动窗口处理长历史数据，不增加模型输入尺寸。
- 模型学习长期用户偏好能力显著提升，商品质量也更好。
- 适合需要处理海量历史数据的推荐系统开发者。
长期运行的推荐系统通常拥有跨越数年的用户-物品交互历史。为有效学习长期用户偏好，大型推荐系统基础模型（FM）需在预训练阶段编码这些信息。传统方法要么生成足够长的序列以包含全部历史，导致模型输入维度过大；要么丢弃部分历史数据，以满足生产环境对模型大小和延迟的要求。本文提出一种滑动窗口训练技术，在不增加模型输入维度的前提下，将长用户历史序列引入训练过程。实验表明，该方法在定量与定性层面均显著提升了推荐系统基础模型对用户长期偏好的学习能力，同时预训练阶段学到的物品目录平均质量也有所提高。
原文摘要 · Abstract (English)
Long-lived recommender systems (RecSys) often encounter lengthy user-item interaction histories that span many years. To effectively learn long term user preferences, Large RecSys foundation models (FM) need to encode this information in pretraining. Usually, this is done by either generating a long enough sequence length to take all history sequences as input at the cost of large model input dimension or by dropping some parts of the user history to accommodate model size and latency requirements on the production serving side. In this paper, we introduce a sliding window training technique to incorporate long user history sequences during training time without increasing the model input dimension. We show the quantitative & qualitative improvements this technique brings to the RecSys FM in learning user long term preferences. We additionally show that the average quality of items in the catalog learnt in pretraining also improves.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。