arXiv:2509.06314cs.LGcs.CV2025-09被引 2

提出新指标ρ(C)直接衡量隐空间冗余,揭示模型效率瓶颈。

Evaluating the Efficiency of Latent Spaces via the Coupling-Matrix

  • 基于耦合矩阵分析维度间依赖,用能量距离量化冗余程度。
  • ρ(C)越低,分类准确率越高或重建误差越小,性能更稳定。
  • 可指导神经网络架构搜索,适合关注模型效率的研究者。

表示学习的核心挑战在于构建既具表达力又高效的隐空间。实践中,深度网络常产生冗余隐空间,多个坐标编码重叠信息,降低有效容量并阻碍泛化。标准指标如准确率或重建损失仅提供间接证据,无法分离冗余这一失败模式。本文引入冗余指数ρ(C),通过分析隐表示导出的耦合矩阵,比较其非对角统计量与正态分布的能量距离,直接量化维度间依赖。该指标紧凑、可解释且统计基础坚实。我们在MNIST变体、Fashion-MNIST、CIFAR-10和CIFAR-100上验证了ρ(C),涵盖多种架构与超参数优化策略。实验表明,低ρ(C)可靠预测高分类准确率或低重建误差,而冗余升高则伴随性能崩溃。估计器可靠性随隐空间维度增加,自然给出可靠分析的下限。我们进一步发现树状帕尔岑估计算法(TPE)优先探索低ρ区域,表明ρ(C)可用于引导神经架构搜索,并作为冗余感知正则化目标。ρ(C)揭示冗余是跨模型与任务的普遍瓶颈,为评估与改进学习表示效率提供了理论视角与实用工具。

原文摘要 · Abstract (English)

A central challenge in representation learning is constructing latent embeddings that are both expressive and efficient. In practice, deep networks often produce redundant latent spaces where multiple coordinates encode overlapping information, reducing effective capacity and hindering generalization. Standard metrics such as accuracy or reconstruction loss provide only indirect evidence of such redundancy and cannot isolate it as a failure mode. We introduce a redundancy index, denoted rho(C), that directly quantifies inter-dimensional dependencies by analyzing coupling matrices derived from latent representations and comparing their off-diagonal statistics against a normal distribution via energy distance. The result is a compact, interpretable, and statistically grounded measure of representational quality. We validate rho(C) across discriminative and generative settings on MNIST variants, Fashion-MNIST, CIFAR-10, and CIFAR-100, spanning multiple architectures and hyperparameter optimization strategies. Empirically, low rho(C) reliably predicts high classification accuracy or low reconstruction error, while elevated redundancy is associated with performance collapse. Estimator reliability grows with latent dimension, yielding natural lower bounds for reliable analysis. We further show that Tree-structured Parzen Estimators (TPE) preferentially explore low-rho regions, suggesting that rho(C) can guide neural architecture search and serve as a redundancy-aware regularization target. By exposing redundancy as a universal bottleneck across models and tasks, rho(C) offers both a theoretical lens and a practical tool for evaluating and improving the efficiency of learned representations.

表示学习冗余度评估隐空间分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。