arXiv:2602.08105cs.LGphysics.data-an2026-02被引 3

用信息瓶颈法精准估算任务相关隐空间维度,避免传统方法高估。

Mutual information and task-relevant latent dimensionality

  • 将任务维度估计转为信息瓶颈问题,用混合判别器保留隐空间几何结构。
  • 在合成数据上准确恢复已知任务维度,噪声环境下仍稳定可靠。
  • 适用于物理数据集,可替代传统几何方法,适合需要降维的科研场景。

估计预测所需的潜在表示维度——任务相关维度——是一个广泛存在但尚未解决的难题。本文将其建模为信息瓶颈问题:在压缩预测器与被预测视图的同时,保持其互信息(MI)的最小嵌入瓶颈维度是多少。该方法复用神经网络互信息估计算法进行维度估计。我们发现,使用可分离/双线性判别器的标准神经估计算法会系统性高估维度,为此提出一种混合判别器,在保留显式维度瓶颈的同时允许灵活的非线性跨视图交互,从而保持潜在空间几何结构。进一步提出单次协议,仅需一个过参数化混合模型即可读取有效维度,无需对瓶颈尺寸进行遍历。我们在具有已知任务相关维度的合成问题上验证了该方法的有效性。通过构建单一数据集的成对视图,将方法扩展至内在维度估计,实现与经典几何维度估计算法的对比。在噪声条件下,传统方法性能下降时,本方法依然可靠。最后,我们在多个物理数据集上展示了该方法的实用性。

原文摘要 · Abstract (English)

Estimating the dimensionality of the latent representation needed for prediction -- the task-relevant dimension -- is a difficult, largely unsolved problem with broad scientific applications. We cast it as an Information Bottleneck question: what embedding bottleneck dimension is sufficient to compress predictor and predicted views while preserving their mutual information (MI). This repurposes neural MI estimators for dimensionality estimation. We show that standard neural estimators with separable/bilinear critics systematically inflate the inferred dimension, and we address this by introducing a hybrid critic that retains an explicit dimensional bottleneck while allowing flexible nonlinear cross-view interactions, thereby preserving the latent geometry. We further propose a one-shot protocol that reads off the effective dimension from a single over-parameterized hybrid model, without sweeping over bottleneck sizes. We validate the approach on synthetic problems with known task-relevant dimension. We extend the approach to intrinsic dimensionality by constructing paired views of a single dataset, enabling comparison with classical geometric dimension estimators. In noisy regimes where those estimators degrade, our approach remains reliable. Finally, we demonstrate the utility of the method on multiple physics datasets.

信息瓶颈维度估计隐空间神经估计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。