arXiv:2511.02101cs.LGcs.IT2025-11被引 13

首次测量地球表示的内在维度,发现其信息量远低于表面复杂度。

Measuring the Intrinsic Dimension of Earth Representations

  • 用内在维数衡量地理隐式神经表示的信息容量
  • 256-512维模型的内在维数仅2-10,随分辨率和模态变化
  • 内在维数与下游任务表现相关,可用于无监督模型评估

在地球观测表征学习中,地理隐式神经表示(INRs)将低维位置输入(经度、纬度)映射到高维嵌入空间,通过训练卫星图像或文本数据实现。尽管目标是压缩地球数据为紧凑可学习表征,但对其包含多少信息、信息集中于何处仍缺乏理解。内在维数衡量数据局部变异所需的自由度,与嵌入空间维度无关。本文首次研究地理INRs的内在维度,分析了256至512维的INRs,发现其内在维数约为2至10,且对预训练时的空间分辨率和输入模态敏感。进一步表明,地理INR的内在维数与下游任务性能相关,并能捕捉空间伪影,有助于模型评估与诊断。本工作提供了一种与架构无关、无需标签的信息量度量,可用于无监督评估、模型选择及预训练设计。

原文摘要 · Abstract (English)

Within the context of representation learning for Earth observation, geographic Implicit Neural Representations (INRs) embed low-dimensional location inputs (longitude, latitude) into high-dimensional embeddings, through models trained on geo-referenced satellite, image or text data. Despite the common aim of geographic INRs to distill Earth's data into compact, learning-friendly representations, we lack an understanding of how much information is contained in these Earth representations, and where that information is concentrated. The intrinsic dimension of a dataset measures the number of degrees of freedom required to capture its local variability, regardless of the ambient high-dimensional space in which it is embedded. This work provides the first study of the intrinsic dimensionality of geographic INRs. Analyzing INRs with ambient dimension between 256 and 512, we find that their intrinsic dimensions fall roughly between 2 and 10 and are sensitive to changing spatial resolution and input modalities during INR pre-training. Furthermore, we show that the intrinsic dimension of a geographic INR correlates with downstream task performance and can capture spatial artifacts, facilitating model evaluation and diagnostics. More broadly, our work offers an architecture-agnostic, label-free metric of information content that can enable unsupervised evaluation, model selection, and pre-training design across INRs.

地理表示内在维度无监督评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。