破解生成式AI在科学中的幻觉问题，揭示其如何通过工作流实现可靠知识产出。
From Hallucination to Reliability: Generative Modeling and the Structure of Scientific Inference
- 将幻觉视为模型生成过程引入的非策略性误表征，而非数据继承
- 案例分析表明，科学工作流可过滤幻觉内容，防止错误传播
- 工作流本身是独立的认知评价单元，可提升推理可靠性
生成式AI在科学中应用日益广泛，但不可避免存在幻觉问题。本文提出一种可靠性理论，解释生成式AI如何产生新科学知识。将幻觉定义为模型生成活动中引入的非策略性目标现象误表征，而非源自训练数据。通过对AlphaFold和SEEDS的案例研究，表明科学工作流利用对目标现象的既有知识，对生成结果进行筛选或修正，从而阻止错误内容进入下游推理。最后指出，工作流本身构成了独立的认知评价单位。
原文摘要 · Abstract (English)
Generative AI is increasingly used in science, but is unavoidably prone to hallucination. I develop a reliabilist account of how generative AI nevertheless gives rise to new scientific knowledge. I analyze hallucinations as non-strategic misrepresentations of the target phenomenon, introduced by a model's generative activity, rather than inherited from training data. Through case studies of AlphaFold and SEEDS, I show how scientific workflows draw on pre-existing knowledge of target phenomena to filter or qualify hallucinatory outputs, thereby preventing their erroneous content from propagating into downstream inference. Finally, I show that workflows are units of epistemic evaluation in their own right.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。