6.5%的神经符号文献可复现，研究提出六阶段审计框架。
6.5% of the Neuro-Symbolic Literature Can Be Reproduced from Its Published Artifacts, a Six-Stage Audit Framework and First Instantiation
- 构建六阶段复现审计框架，覆盖文献检索到结果验证全流程。
- 仅85篇（6.5%）论文成功复现，多数因缺少代码或非代码数据缺失阻塞。
- 呼吁论文投稿时强制提交完整、版本化、永久存档的代码与数据包。
本文提出一种六阶段框架，用于审计计算机科学领域内科研结论的可复现性，并首次在神经符号人工智能（NSAI）子领域进行实例化。审计历时多年：第一阶段检索5,497条记录，去重后保留2,479条；第二阶段筛选出1,365篇自认属于NSAI的文献，剔除61篇无关或无定量评估的论文；第三阶段为剩余1,304篇论文寻找可验证的公开代码，发现849篇无代码，仅455篇进入后续流程。最终对其中85篇完成完全或部分复现，占可复现论文的6.52%（总候选集的18.68%）。另有321次尝试因缺少非代码类数据受阻，42次因代码库缺失或不可用失败。这些数字揭示了即便宣称“代码可用”，复现仍存在持续缺口，亟需未来发表时强制提供完整、版本化、永久归档的成果包。
原文摘要 · Abstract (English)
We present a six-stage framework for auditing the reproducibility of scientific claims across a research literature within the computer science domain, and instantiate our framework for the neuro-symbolic AI (NSAI) subdomain. Instantiating the framework on the NSAI subdomain produced a multi-year audit. Stage one retrieved 5,497 records and removed 3,018 duplicates. Stage two screened the 2,479 unique records at title and abstract, identifying 1,365 self-identified NSAI records, then removed a further 61 at full text for off-topic, non-research, no-quantitative-evaluation, or inaccessible-full-text reasons. Stage three sought a verifiable public code artifact for each of the 1,304 eligible records and found none for 849, leaving 455 to enter the artifact inventory and bounded rerun of stages four and five. We fully or partially reproduced 85 studies, 6.52% of the eligible corpus and 18.68% of attempted reruns. We found that 321 attempted reruns were blocked by missing non- code artifacts and 42 by missing or unusable code repositories. These figures quantify a persistent reproducibility deficit that survives even nominal "code available" declarations, and signal the need for enforced, versioned, and permanently archived artifact bundles in future NSAI publications. We argue that empirical NSAI papers should be required at submission time to provide complete, versioned, and permanently archived artifact bundles.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。