厘清大模型是否真‘知道’，从哲学定义出发构建评估框架
Defining Knowledge: Bridging Epistemology and Large Language Models
- 基于认识论定义，形式化大模型知识的可解释标准
- 100位专家调研显示对‘知识’定义存在显著分歧
- 提出符合主流哲学定义的知识评估协议，适合研究者参考
大语言模型相关文献中充斥着关于‘知识’的宣称，但能否说 GPT-4 真正‘知道’地球是圆的？本文回顾认识论中知识的标准定义，并将其形式化为适用于大模型的解释框架。研究发现当前自然语言处理领域在知识概念上与认识论框架存在不一致和空白。此外，我们对100位专业哲学家和计算机科学家进行了调研，比较他们对知识定义的偏好及对大模型是否能真正‘知道’的看法。最后，我们提出了依据最相关定义设计的知识测试评估协议。
原文摘要 · Abstract (English)
Knowledge claims are abundant in the literature on large language models (LLMs); but can we say that GPT-4 truly "knows" the Earth is round? To address this question, we review standard definitions of knowledge in epistemology and we formalize interpretations applicable to LLMs. In doing so, we identify inconsistencies and gaps in how current NLP research conceptualizes knowledge with respect to epistemological frameworks. Additionally, we conduct a survey of 100 professional philosophers and computer scientists to compare their preferences in knowledge definitions and their views on whether LLMs can really be said to know. Finally, we suggest evaluation protocols for testing knowledge in accordance to the most relevant definitions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。