用信息论指标量化生成模型的可解释性，评估隐变量解耦程度。
Analyzing Generative Models by Manifold Entropic Metrics
- 基于独立机制原理设计可计算的信息论评估指标
- 在EMNIST上对比多种流模型与β-VAE，发现训练方式影响解耦效果
- 能排序隐变量重要性并检测残留相关性，适合模型可解释性研究
优秀的生成模型不仅应生成高质量数据，还应具备可解释的表示以帮助理解其行为。然而，难以客观衡量解耦表示的理想特性是否达成。受独立机制原则启发，我们提出一组可计算的信息论评估指标，用于解决这一难题。我们在典型玩具例子上验证了方法有效性，并在EMNIST数据集上对多种归一化流架构和β-VAE进行了深入比较。该方法可对隐变量按重要性排序，并评估所得概念间的残余相关性。实验中最有趣的发现是：不同模型架构与训练流程在促使模型收敛到对齐且解耦表示方面具有不同的归纳偏置，可据此进行排名。
原文摘要 · Abstract (English)
Good generative models should not only synthesize high quality data, but also utilize interpretable representations that aid human understanding of their behavior. However, it is difficult to measure objectively if and to what degree desirable properties of disentangled representations have been achieved. Inspired by the principle of independent mechanisms, we address this difficulty by introducing a novel set of tractable information-theoretic evaluation metrics. We demonstrate the usefulness of our metrics on illustrative toy examples and conduct an in-depth comparison of various normalizing flow architectures and $β$-VAEs on the EMNIST dataset. Our method allows to sort latent features by importance and assess the amount of residual correlations of the resulting concepts. The most interesting finding of our experiments is a ranking of model architectures and training procedures in terms of their inductive bias to converge to aligned and disentangled representations during training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。