arXiv:2507.16289cs.IRcs.LG2025-07中稿 · ACM RecSys 2025被引 27

对比不同数据划分策略,发现评估结果差异大,影响模型排序与实际部署。

Time to Split: Exploring Data Splitting Strategies for Offline Evaluation of Sequential Recommenders

  • 系统比较多种数据划分方法在序列推荐中的表现
  • 留一法存在时间泄漏问题,评估结果不真实
  • 适合关注评估公平性与实际应用的算法研究者

现代序列推荐系统(如基于Transformer或大语言模型的模型)在下一物品预测任务中表现优异,但其评估协议仍不完善:常见方法如留一法虽匹配预测任务,却允许训练与测试期重叠,导致时间泄漏和过长测试周期,降低现实相关性。全局时间划分通过在独立未来时间段评估解决了这些问题,但在目标交互选择和验证集构建方面仍缺乏明确规范。本文系统比较了多种划分策略在多个数据集和基准模型上的表现,发现不同策略会导致显著不同的评估结果,影响模型排名与实际部署决策。研究强调需提升学术与工业界评估的可复现性。

原文摘要 · Abstract (English)

Modern sequential recommender systems, ranging from lightweight transformer-based variants to large language models, have become increasingly prominent in academia and industry due to their strong performance in the next-item prediction task. Yet common evaluation protocols for sequential recommendations remain insufficiently developed: they often fail to reflect the corresponding recommendation task accurately, or are not aligned with real-world scenarios. Although the widely used leave-one-out split matches next-item prediction, it permits the overlap between training and test periods, which leads to temporal leakage and unrealistically long test horizon, limiting real-world relevance. Global temporal splitting addresses these issues by evaluating on distinct future periods. However, its applications to sequential recommendations remain loosely defined, particularly in terms of selecting target interactions and constructing a validation subset that provides necessary consistency between validation and test metrics. In this paper, we demonstrate that evaluation outcomes can vary significantly across splitting strategies, influencing model rankings and practical deployment decisions. To improve reproducibility in both academic and industrial settings, we systematically compare different splitting strategies for sequential recommendations across multiple datasets and established baselines. Our findings show that prevalent splits, such as leave-one-out, may be insufficiently aligned with more realistic evaluation strategies. Code: https://github.com/monkey0head/time-to-split

序列推荐数据划分评估方法时间泄漏

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。