解决高维数据中互信息估计的数值溢出问题
Improving Numerical Stability of Normalized Mutual Information Estimator on High Dimensions
- 采用对数变换稳定高维空间中的k-NN半径计算
- 避免数值溢出，保持估计精度和方差稳定
- 适合处理数百维以上数据的互信息分析任务
互信息是衡量变量间共享信息量的强大通用度量。基于k近邻（k-NN）的归一化互信息估计涉及尺度不变k-NN半径的计算，当数据联合维度较高（通常达数百维）时，该计算易发生数值溢出。为此，本文提出一种对数变换技术，通过在半径计算中引入该变换，有效避免数值溢出，同时保持计算精度。理论分析与实证评估均表明，该方法在不增加显著计算开销、不引入额外偏差或破坏估计方差的前提下，显著提升了高维场景下的数值稳定性。
原文摘要 · Abstract (English)
Mutual information provides a powerful, general-purpose metric for quantifying the amount of shared information between variables. Estimating normalized mutual information using a k-Nearest Neighbor (k-NN) based approach involves the calculation of the scaling-invariant k-NN radius. Calculation of the radius suffers from numerical overflow when the joint dimensionality of the data becomes high, typically in the range of several hundred dimensions. To address this issue, we propose a logarithmic transformation technique that improves the numerical stability of the radius calculation in high-dimensional spaces. By applying the proposed transformation during the calculation of the radius, numerical overflow is avoided, and precision is maintained. Proposed transformation is validated through both theoretical analysis and empirical evaluation, demonstrating its ability to stabilize the calculation without compromising precision, increasing bias, or adding significant computational overhead, while also helping to maintain estimator variance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。