arXiv:2607.26074cs.IRcs.HC2026-07中稿 · ACM Transactions o…综述

分析51篇推荐系统可复现论文,揭示研究透明化现状与挑战。

Reproducibility in Recommender Systems: A Survey

  • 系统梳理2020-2025年可复现论文的方法与数据模式
  • 发现多数研究使用有限数据集和评估协议
  • 适合关注科研可信度与实验规范的研究者阅读

可复现性已成为推荐系统研究可信性的基石,源于对实验结果可靠性与泛化能力的日益关注。为此,ACM RecSys会议自2020年起设立可复现性专题,推动严谨、透明、可重复的研究。本文对2020至2025年共51篇入选论文进行结构化分析,按贡献类型分类,并探讨数据集、算法、框架与评估实践的常见模式,旨在理解社区中可复现性的实际操作方式。研究发现三大趋势:一是专题范围扩大,从单纯复现扩展至基准测试、资源建设与方法论贡献;二是可复现论文呈现一致的方法学特征,依赖少数数据集、算法与评估协议;三是实践中常通过扩展已有实验而非严格复制来实现可复现,研究多引入新模型或评估指标。总体而言,可复现工作提升了透明度与文档规范性,但对方法多样性影响有限,暴露出概念定义与实际执行之间的差距。

原文摘要 · Abstract (English)

Reproducibility has become a cornerstone of credible recommender systems research, driven by growing concerns about the reliability and generalizability of experimental results. In response, the ACM RecSys conference introduced a dedicated Reproducibility Track in 2020 to encourage rigorous, transparent, and repeatable research. This paper presents a structured analysis of the track from 2020 to 2025, covering 51 accepted papers. We classify contributions by type and analyze common patterns in datasets, algorithms, frameworks, and evaluation practices, with the goal of understanding how reproducibility is operationalized in practice within the community. Our findings reveal three main trends. First, the track has expanded in scope, evolving from a focus on reproduction and replication to include benchmarking, resources, and methodological contributions. Second, reproducibility papers exhibit a consistent methodological profile, relying on a limited set of datasets, algorithms, and evaluation protocols. Third, reproducibility in practice often involves extending prior experiments rather than strictly replicating them, with studies frequently introducing additional models or evaluation criteria. Overall, reproducibility work has improved transparency and documentation, but has had limited impact on methodological diversity, highlighting a gap between the conceptual definition of reproducibility and its implementation.

可复现性推荐系统研究规范

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。