LLM易被伪科学误导，生成看似合理实则错误的科学回答。
Large language models eroding science understanding: an experimental study
- 用特定边缘论文训练模型，使其优先采纳非主流观点。
- 修改后的模型给出流畅且违背科学共识的答案，难以识别为错误。
- 警示公众与教育者：不能轻信AI的科学解释，需专家把关。
本研究在人工智能与伦理领域评审中，考察大型语言模型（LLMs）在回答科学问题时的可靠性，并揭示其易受边缘科学内容影响的弱点。研究者对定制化LLM进行改造，使其优先采纳关于精细结构常数和引力波的特定边缘论文知识，随后与领域专家及标准LLM的回答进行对比。结果显示，经修改的模型生成了语言流畅、逻辑自洽但违背科学共识的答案，且非专业人士难以辨别其误导性。该研究证明LLM存在可被操纵的风险，无法替代专家判断，凸显其在公众科学认知传播中可能引发的信息谬误风险。
原文摘要 · Abstract (English)
This paper is under review in AI and Ethics This study examines whether large language models (LLMs) can reliably answer scientific questions and demonstrates how easily they can be influenced by fringe scientific material. The authors modified custom LLMs to prioritise knowledge in selected fringe papers on the Fine Structure Constant and Gravitational Waves, then compared their responses with those of domain experts and standard LLMs. The altered models produced fluent, convincing answers that contradicted scientific consensus and were difficult for non-experts to detect as misleading. The results show that LLMs are vulnerable to manipulation and cannot replace expert judgment, highlighting risks for public understanding of science and the potential spread of misinformation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。