为图像和文本数据设计了评估神经互信息估计器的基准套件。
A Benchmark Suite for Evaluating Neural Mutual Information Estimators on Unstructured Datasets
- 用同类别采样和二元对称信道操控真实数据的互信息值。
- 在7个挑战场景中验证了神经互信息估计器的可靠性。
- 适合研究互信息估计与复杂数据建模的学者参考。
互信息（MI）是衡量两个随机变量依赖关系的基本指标。当仅能获取样本而无法获得底层分布函数时，可通过基于样本的估计方法评估MI。然而，现有对MI估计器的评估几乎都依赖于高斯多元等解析数据集，虽可解析计算真实MI值，但难以反映真实数据的复杂性。本文提出一个针对图像和文本等非结构化数据的综合性基准套件，通过同类别采样实现正样本配对，并引入二元对称信道技巧，精确调控真实数据的理论MI值。利用该套件，我们分析了七个挑战性场景，揭示了神经互信息估计器在非结构化数据上的表现可靠性。
原文摘要 · Abstract (English)
Mutual Information (MI) is a fundamental metric for quantifying dependency between two random variables. When we can access only the samples, but not the underlying distribution functions, we can evaluate MI using sample-based estimators. Assessment of such MI estimators, however, has almost always relied on analytical datasets including Gaussian multivariates. Such datasets allow analytical calculations of the true MI values, but they are limited in that they do not reflect the complexities of real-world datasets. This study introduces a comprehensive benchmark suite for evaluating neural MI estimators on unstructured datasets, specifically focusing on images and texts. By leveraging same-class sampling for positive pairing and introducing a binary symmetric channel trick, we show that we can accurately manipulate true MI values of real-world datasets. Using the benchmark suite, we investigate seven challenging scenarios, shedding light on the reliability of neural MI estimators for unstructured datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。