用智能体框架自动提出并验证社会科学研究假设,效率提升2-4倍。
Accelerating Social Science Research via Agentic Hypothesization and Experimentation
- 分两阶段用生成器提假设、实验器实证,类贝叶斯优化加速发现
- 新假设数量多2-4倍,预测力高7-17%,在多模态数据上也适用
- 专家评价88%新颖,70%有潜力,首个验证大模型假说的A/B测试
基于数据的社会科学研宄本质上进展缓慢,依赖观察、假设生成与实验验证的迭代循环。尽管近期数据驱动方法能加速部分流程,却难以支持端到端科学发现。为此,我们提出EXPERIGEN——一个基于智能体的框架,通过受贝叶斯优化启发的两阶段搜索,实现端到端发现:生成器提出候选假设,实验器进行实证评估。在多个领域中,EXPERIGEN持续发现2-4倍更多统计显著的假设,且预测能力比先前方法高出7-17%。该框架自然扩展至多模态和关系型数据。除统计性能外,假设还需具备新颖性、实证基础与可行动性。我们邀请资深教师对25个机器生成假设进行评审,其中88%被评作中等或高度新颖,70%被认为具影响力且值得深入研究,多数展现出与高级研究生研究相当的严谨性。最后,为验证真实世界有效性,我们首次开展大模型生成假设的A/B测试,结果显示显著性p < 1e-6,效应量高达344%。
原文摘要 · Abstract (English)
Data-driven social science research is inherently slow, relying on iterative cycles of observation, hypothesis generation, and experimental validation. While recent data-driven methods promise to accelerate parts of this process, they largely fail to support end-to-end scientific discovery. To address this gap, we introduce EXPERIGEN, an agentic framework that operationalizes end-to-end discovery through a Bayesian optimization inspired two-phase search, in which a Generator proposes candidate hypotheses and an Experimenter evaluates them empirically. Across multiple domains, EXPERIGEN consistently discovers 2-4x more statistically significant hypotheses that are 7-17 percent more predictive than prior approaches, and naturally extends to complex data regimes including multimodal and relational datasets. Beyond statistical performance, hypotheses must be novel, empirically grounded, and actionable to drive real scientific progress. To evaluate these qualities, we conduct an expert review of machine-generated hypotheses, collecting feedback from senior faculty. Among 25 reviewed hypotheses, 88 percent were rated moderately or strongly novel, 70 percent were deemed impactful and worth pursuing, and most demonstrated rigor comparable to senior graduate-level research. Finally, recognizing that ultimate validation requires real-world evidence, we conduct the first A/B test of LLM-generated hypotheses, observing statistically significant results with p less than 1e-6 and a large effect size of 344 percent.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。