统计学习未必带来真知识，数据不足时无法完全掌握真相。
Statistical learning does not always entail knowledge
- 用贝叶斯方法建模信念更新，数据约束导致后验呈吉布斯分布。
- 特征提取过少时，学习不完全；知识获取永远无法彻底完成。
- 区分主次学习，他人学习信息不等于真知识，适合反思算法局限者。
本文研究智能体对真假命题的学习与知识获取（LKA）问题。采用贝叶斯框架，智能体根据接收到的数据更新关于命题的信念，形成后验分布。知识获取以主动信息衡量，数据作为外部信息影响信念。假设数据揭示与命题相关的若干特征，推导出后验为在特征约束下最大熵的吉布斯分布。研究表明：当提取特征数量过少时，完全学习不可能实现；而无论数据多少，完全知识获取始终不可达。进一步区分初级学习（获取相关特征数据）与次级学习（获取他人学习结果），并指出后者不构成真实知识获取。理论结果对统计学习算法有启示：此类算法并不总能产生真正知识。文中通过多个实例进行说明。
原文摘要 · Abstract (English)
In this paper, we study learning and knowledge acquisition (LKA) of an agent about a proposition that is either true or false. We use a Bayesian approach, where the agent receives data to update his beliefs about the proposition according to a posterior distribution. The LKA is formulated in terms of active information, with data representing external or exogenous information that modifies the agent's beliefs. It is assumed that data provide details about a number of features that are relevant to the proposition. We show that this leads to a Gibbs distribution posterior, which is in maximum entropy relative to the prior, conditioned on the side constraints that the data provide in terms of the features. We demonstrate that full learning is sometimes not possible and full knowledge acquisition is never possible when the number of extracted features is too small. We also distinguish between primary learning (receiving data about features of relevance for the proposition) and secondary learning (receiving data about the learning of another agent). We argue that this type of secondary learning does not represent true knowledge acquisition. Our results have implications for statistical learning algorithms, and we claim that such algorithms do not always generate true knowledge. The theory is illustrated with several examples.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。