用学习到的特征表示计算信息度量，提升效率与理论保障
Separable Computation of Information Measures
- 通过特征表示实现信息度量的可分离计算
- 涵盖互信息、信息瓶颈等五类核心度量
- 为表示学习中的信息估计提供理论支撑
我们研究了信息度量的可分离计算设计，即从学习到的特征表示而非原始数据中计算信息度量。在对特征表示施加弱假设的前提下，证明一类信息度量（包括互信息、f-信息、Wyner公共信息、Gács-Körrner公共信息以及Tishby信息瓶颈）均支持这种可分离计算。该研究揭示了信息度量与统计依赖结构之间的若干新联系，并为通过表示学习估计信息度量的实际设计提供了理论保证。
原文摘要 · Abstract (English)
We study a separable design for computing information measures, where the information measure is computed from learned feature representations instead of raw data. Under mild assumptions on the feature representations, we demonstrate that a class of information measures admit such separable computation, including mutual information, $f$-information, Wyner's common information, G{á}cs--K{ö}rner common information, and Tishby's information bottleneck. Our development establishes several new connections between information measures and the statistical dependence structure. The characterizations also provide theoretical guarantees of practical designs for estimating information measures through representation learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。