十年间AI论文的开源与可复现性显著提升，代码数据共享率翻了近六倍。
The Shift Toward Open and Reproducible AI Research
- 分析56800篇顶会论文，量化7项可复现性指标变化
- 代码数据共享率从11%升至64%，估算可复现率从28%增至64%
- 改进早于清单制度推行，反映开放科学趋势而非政策驱动
AI研究中的可复现性危机促使学术界改善文档规范。多项研究指出了方法论问题，为此领域内最具影响力的会议引入了可复现性检查清单。本文通过评估过去十年五大顶级AI会议的所有发表论文，探究文档实践是否随时间演变。我们识别出七项可复现性变量，经质量验证后分析了56,800篇论文。结果表明，2014至2024年间，文档实践明显改善：同时公开代码与数据的论文比例从11%增至64%。基于前期研究的实证可复现率，我们推断——并非直接测试，而是依据文档实践——可复现性从2014年的28%提升至2024年的64%。文档改进出现在检查清单推出之前，说明这些变化反映了更广泛的开放科学趋势，而非对正式要求的直接响应。
原文摘要 · Abstract (English)
The reproducibility crisis has directed the AI research community toward improving documentation practices. Several studies have identified methodological issues, and in response, the most impactful venues in the field have introduced reproducibility checklists. We seek to understand whether documentation practices have changed over time by assessing all published papers at five leading AI conferences over the past decade. Seven reproducibility variables were identified, quality-assured and used to analyse 56 800 publications. Our analysis reveals that in the period 2014 to 2024, documentation practices have improved; papers sharing both code and data increased nearly sixfold, from 11% to 64% Building on empirical reproducibility rates from a prior study, we estimate - inferred from documentation practices, not direct testing - that reproducibility increased from 28% in 2014 to 64% in 2024. Improvements in documentation practices predate the introduction of reproducibility checklists, suggesting these changes reflect a broader movement toward open science rather than a direct response to formal requirements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。