揭示切片互信息在依赖性测量中的欺骗性缺陷
Curse of Slicing: Why Sliced Mutual Information is a Deceptive Measure of Statistical Dependence
- 通过理论与实验发现切片互信息易饱和且对数据操纵敏感
- 检测依赖性增强时表现失灵，冗余信息反而被优先捕捉
- 适合关注高维依赖性度量可靠性的研究者阅读
切片互信息（SMI）被广泛用作衡量非线性统计依赖性的可扩展替代方案，具备收敛快、对高维数据鲁棒、仅在统计独立时为零等优点。然而，我们通过大量基准测试和理论分析表明，SMI极易饱和，无法有效检测依赖性增强，优先捕获冗余信息而非有用内容，在某些情况下甚至劣于皮尔逊相关系数。这些反直觉行为使其在实际应用中存在严重误导风险。
原文摘要 · Abstract (English)
Sliced Mutual Information (SMI) is widely used as a scalable alternative to mutual information for measuring non-linear statistical dependence. Despite its advantages, such as faster convergence, robustness to high dimensionality, and nullification only under statistical independence, we demonstrate that SMI is highly susceptible to data manipulation and exhibits counterintuitive behavior. Through extensive benchmarking and theoretical analysis, we show that SMI saturates easily, fails to detect increases in statistical dependence, prioritizes redundancy over informative content, and in some cases, performs worse than correlation coefficient.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。