hierarchical 模型的先验为何具有信息依赖性，用最大熵原理揭示其内在机制。
Bayesian Hierarchical Models and the Maximum Entropy Principle
- 用最大熵分布构造层次先验，通过超参数控制参数间依赖关系。
- 边缘先验仍具最大熵性质，约束条件变为对未知量函数分布的限制。
- 揭示了层次建模中隐含的统计假设，适合概率建模研究者阅读。
层次贝叶斯模型在实际数据分析中广泛应用。一种理解是，它们通过引入超参数，间接为未知参数设定先验。由此产生的参数边缘先验（对超参数积分）通常存在依赖性，即学习一个参数会提供其他参数的信息。本文证明：当给定超参数的先验为规范分布（即在矩约束下的最大熵分布）时，参数的边缘先验同样具有最大熵性质，但约束条件针对的是未知量的某个函数的边际分布。这一结果揭示了在设定层次模型时实际上所假设的信息结构。
原文摘要 · Abstract (English)
Bayesian hierarchical models are frequently used in practical data analysis contexts. One interpretation of these models is that they provide an indirect way of assigning a prior for unknown parameters, through the introduction of hyperparameters. The resulting marginal prior for the parameters (integrating over the hyperparameters) is usually dependent, so that learning one parameter provides some information about the others. In this contribution, I will demonstrate that, when the prior given the hyperparameters is a canonical distribution (a maximum entropy distribution with moment constraints), the dependent marginal prior also has a maximum entropy property, with a different constraint. This constraint is on the marginal distribution of some function of the unknown quantities. The results shed light on what information is actually being assumed when we assign a hierarchical model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。