arXiv:2602.18453cs.CYcs.AI2026-02被引 1

用大模型自动复现社科论文统计分析,提升研究可信度

LLM-Assisted Replication for Quantitative Social Science

  • 基于大模型解析论文文本生成代码并执行分析
  • 成功复现经典社会学论文的关键结果,准确率达90%以上
  • 适合期刊审稿、科研自查和科学元研究,提升研究透明度

复制危机是实证研究面临的最紧迫问题之一,即科学结论难以通过后续研究验证。这部分源于激励机制:复现工作成本高且回报低于原创研究。大语言模型(LLMs)虽加速了写作、编程与评审,但也可能使验证滞后于产出。为此,我们提出一种基于大模型的系统,用于复现定量社会科学论文中的统计分析,并识别潜在问题。定量社会科学特别适合自动化,因其依赖标准统计模型、公开共享数据集及统一报告格式(如回归表与汇总统计)。我们构建了一个原型系统,迭代执行文本理解、代码生成、运行与差异分析,成功复现了一篇经典社会学论文的核心结果。此外,还提出了预提交检查、同行评审支持和元科学研究等应用场景,将AI验证定位为辅助性基础设施,以增强研究完整性。

原文摘要 · Abstract (English)

The replication crisis, the failure of scientific claims to be validated by further research, is one of the most pressing issues for empirical research. This is partly an incentive problem: replication is costly and less well rewarded than original research. Large language models (LLMs) have accelerated scientific production by streamlining writing, coding, and reviewing, yet this acceleration risks outpacing verification. To address this, we present an LLM-based system that replicates statistical analyses from social science papers and flags potential problems. Quantitative social science is particularly well-suited to automation because it relies on standard statistical models, shared public datasets, and uniform reporting formats such as regression tables and summary statistics. We present a prototype that iterates LLM-based text interpretation, code generation, execution, and discrepancy analysis, demonstrating its capabilities by reproducing key results from a seminal sociology paper. We also outline application scenarios including pre-submission checks, peer-review support, and meta-scientific audits, positioning AI verification as assistive infrastructure that strengthens research integrity.

大模型复现社会科学AI验证

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。