用内在维度评估自监督学习表征，更快更准。
IdEst: Assessing Self-Supervised Learning Representations via Intrinsic Dimension
- 通过最小生成树法估算表征的内在维度
- 内在维度与下游线性探针性能强相关
- 可高效选超参数，大幅降低计算成本
自监督学习（SSL）已成为从无标签数据中学习有意义表征的强大范式。然而，评估这些表征的标准方法——线性探针——计算开销大、对超参数敏感，且难以揭示表征空间的几何结构。本文受神经网络泛化与内在维度（ID）关系的启发，提出IdEst方法，利用最小生成树维度估计器（dim_MST）估算SSL表征的内在维度。在多种数据集、模型架构和预训练目标下，我们发现IdEst与下游线性探针性能高度相关。此外，IdEst可实现高效超参数选择，相比监督方法显著降低计算成本。结果表明，内在维度是评估SSL表征的合理几何代理，可补充现有监督探针协议。
原文摘要 · Abstract (English)
Self-supervised learning (SSL) has emerged as a powerful paradigm for learning meaningful representations from unlabeled data. However, the standard protocol for evaluating these representations, linear probing, is computationally expensive, sensitive to hyperparameters, and provides limited insight into the geometric structure of the representation space. In this work, motivated by connections between neural network generalization and intrinsic dimension (ID) we propose IdEst, a method for estimating the ID of SSL representations via the Minimum Spanning Tree dimension estimator ($\mathrm{dim}_\mathrm{MST}$). Across diverse datasets, architectures, and SSL pretraining objectives, we show that IdEst strongly correlates with downstream linear probe performances. Furthermore, we demonstrate that IdEst enables efficient hyperparameter selection, significantly reducing the computational cost compared to supervised alternatives. Our results highlight intrinsic dimensionality as a principled geometric proxy for assessing SSL representations, complementing standard supervised probing protocols.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。