arXiv:2512.23981cs.LG2025-12

提出新指标评估嵌入中信息保留程度,发现传统方法漏检关键信息损失。

Information-Theoretic Quality Metric of Low-Dimensional Embeddings

  • 基于熵与稳定秩设计局部信息保真度度量ERPM
  • ERPM与局部普鲁克雷斯相关性强,但局部差异大
  • 适合金融预警等需精准信息保留的场景

本文从信息论视角研究低维嵌入的质量。传统度量如应力、秩相关或局部普鲁克雷斯仅关注距离或局部几何畸变,未直接衡量高维数据投影至低维时的信息保留情况。为此,提出熵秩保持度量(ERPM),基于邻域矩阵奇异值谱的香农熵与稳定秩,量化原始表示与降维投影间不确定性变化,提供局部指标与全局统计量。在金融时间序列和经典流形上验证,距离基指标与几何及谱度量相关性极低,而ERPM与局部普鲁克雷斯平均相关性强,但在局部区域存在显著差异,表明其可识别严重信息损失的邻域,从而在信息敏感应用(如早期预警系统构建)中补充现有度量,实现更全面评估。

原文摘要 · Abstract (English)

In this work we study the quality of low-dimensional embeddings from an explicitly information-theoretic perspective. We begin by noting that classical evaluation metrics such as stress, rank-based neighborhood criteria, or Local Procrustes quantify distortions in distances or in local geometries, but do not directly assess how much information is preserved when projecting high-dimensional data onto a lower-dimensional space. To address this limitation, we introduce the Entropy Rank Preservation Measure (ERPM), a local metric based on the Shannon entropy of the singular-value spectrum of neighborhood matrices and on the stable rank, which quantifies changes in uncertainty between the original representation and its reduced projection, providing neighborhood-level indicators and a global summary statistic. To validate the results of the metric, we compare its outcomes with the Mean Relative Rank Error (MRRE), which is distance-based, and with Local Procrustes, which is based on geometric properties, using a financial time series and a manifold commonly studied in the literature. We observe that distance-based criteria exhibit very low correlation with geometric and spectral measures, while ERPM and Local Procrustes show strong average correlation but display significant discrepancies in local regimes, leading to the conclusion that ERPM complements existing metrics by identifying neighborhoods with severe information loss, thereby enabling a more comprehensive assessment of embeddings, particularly in information-sensitive applications such as the construction of early-warning indicators.

嵌入质量信息论降维金融分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。