用AI自动验证假设,像科学实验一样找反例,又快又准。
Automated Hypothesis Validation with Agentic Sequential Falsifications
- 用大模型设计反例实验,逐条检验假设的可证伪性。
- 在6个领域验证有效,错误控制严格,效率比人高10倍。
- 适合需要快速、严谨验证复杂假设的研究者。
假设是信息获取、决策和发现的核心,但许多现实中的假设抽象且难以直接验证。随着大语言模型生成大量假设,人工验证已不可行,且易产生幻觉。本文提出Popper框架,基于卡尔·波普尔的可证伪性原则,利用大模型代理设计并执行针对假设可测推论的反例实验。创新的序列化测试机制在严格控制第一类错误的同时,主动从多种观测数据(包括已有数据或新实验)中收集证据。我们在生物学、经济学、社会学等六个领域验证了该方法,结果表明其具备强错误控制能力、高检验功效与良好可扩展性。相较于人类科学家,Popper在验证复杂生物假设时表现相当,但耗时减少10倍,提供了一种可扩展、严谨的假设验证方案。
原文摘要 · Abstract (English)
Hypotheses are central to information acquisition, decision-making, and discovery. However, many real-world hypotheses are abstract, high-level statements that are difficult to validate directly. This challenge is further intensified by the rise of hypothesis generation from Large Language Models (LLMs), which are prone to hallucination and produce hypotheses in volumes that make manual validation impractical. Here we propose Popper, an agentic framework for rigorous automated validation of free-form hypotheses. Guided by Karl Popper's principle of falsification, Popper validates a hypothesis using LLM agents that design and execute falsification experiments targeting its measurable implications. A novel sequential testing framework ensures strict Type-I error control while actively gathering evidence from diverse observations, whether drawn from existing data or newly conducted procedures. We demonstrate Popper on six domains including biology, economics, and sociology. Popper delivers robust error control, high power, and scalability. Furthermore, compared to human scientists, Popper achieved comparable performance in validating complex biological hypotheses while reducing time by 10 folds, providing a scalable, rigorous solution for hypothesis validation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。