用删减法测试AI能否真正发现新知识,而非只是复述旧内容。
Unlearning as Ablation: Toward a Falsifiable Benchmark for Generative Scientific Discovery
- 系统性删除目标结论及其支持内容,检验模型能否重新推导。
- 若无法重推,说明当前AI仍依赖记忆而非创造新知识。
- 适合关注AI科学发现能力与评估方法的研究者。
关于AI在科学中作用的宏大主张——从‘通用人工智能将治愈所有疾病’到大幅加速发现的承诺——引发了一个核心认识论问题:大语言模型(LLMs)是否真正生成新知识,还是仅重组已记忆片段?我们提出‘遗忘即消解’作为可验证的探针,以检验生成性科学发现能力。思路是系统性移除某一目标结果及其遗忘闭包(支持性引理、同义表述、多跳蕴含),再评估模型是否仅基于允许的公理和工具能重新推导该结果。成功意味着超越记忆的生成能力;失败则暴露当前局限。不同于现有遗忘动机(隐私、版权、安全),我们将其重新定位为面向AI for Science的认识论探针。我们在数学与算法领域设计了一个最小可行性原型以证明可行性,并概述该方法未来扩展至物理或化学领域的路径。本文为立场论文,贡献在于概念与方法论,非实证研究。旨在激发讨论:如何通过严谨的消解测试区分重构知识与单纯检索的模型,并引导下一代AI for Science基准的发展。
原文摘要 · Abstract (English)
Bold claims about AI's role in science-from "AGI will cure all diseases" to promises of radically accelerated discovery-raise a central epistemic question: do large language models (LLMs) truly generate new knowledge, or do they merely remix memorized fragments? We propose unlearning-as-ablation as a falsifiable probe of constructive scientific discovery. The idea is to systematically remove a target result together with its forget-closure (supporting lemmas, paraphrases, and multi-hop entailments) and then evaluate whether the model can re-derive the result from only permitted axioms and tools. Success would indicate generative capability beyond recall; failure would expose current limits. Unlike prevailing motivations for unlearning-privacy, copyright, or safety-our framing repositions it as an epistemic probe for AI-for-Science. We outline a minimal pilot in mathematics and algorithms to illustrate feasibility, and sketch how the same approach could later be extended to domains such as physics or chemistry. This is a position paper: our contribution is conceptual and methodological, not empirical. We aim to stimulate discussion on how principled ablation tests could help distinguish models that reconstruct knowledge from those that merely retrieve it, and how such probes might guide the next generation of AI-for-Science benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。