研究遥感模型嵌入设计,发现如何压缩数据仍保持高精度。
How to Embed Matters: Evaluation of EO Embedding Design Choices
- 用不同骨干网络和聚合方式生成遥感嵌入
- 嵌入可压缩至原始数据的1/500且性能不降
- 变压器+均值池化是最佳默认方案,多目标融合更鲁棒
地球观测(EO)任务产生海量多光谱影像,越来越多地通过大型地理空间基础模型(GeoFMs)进行分析。在端到端适配之外,工作流日益依赖中间表示作为任务无关嵌入,实现一次计算、多次复用。因此,当GeoFMs作为特征提取器时,表示获取、聚合与组合方式直接影响下游性能与流程可扩展性。理解这些权衡对构建可扩展的嵌入式EO工作流至关重要——紧凑嵌入可替代原始数据,同时保持广泛适用性。本文基于NeuCo-Bench开展系统性分析,研究骨干架构、预训练策略、表示深度、空间聚合及表示组合对EO任务性能的影响。结果表明,通过聚合可将嵌入压缩至原始输入数据的500倍以下,且仍具备良好可用性。跨模型观察到一致趋势:以均值池化的变压器骨干提供强健默认嵌入;中间层的ResNet表现常优于最终层;自监督目标在特定任务中具优势;结合不同目标的嵌入通常提升鲁棒性。
原文摘要 · Abstract (English)
Earth observation (EO) missions produce petabytes of multispectral imagery, increasingly analyzed using large Geospatial Foundation Models (GeoFMs). Alongside end-to-end adaptation, workflows make growing use of intermediate representations as task-agnostic embeddings, enabling models to compute representations once and reuse them across downstream tasks. Consequently, when GeoFMs act as feature extractors, decisions about how representations are obtained, aggregated, and combined affect downstream performance and pipeline scalability. Understanding these trade-offs is essential for scalable embedding-based EO workflows, where compact embeddings can replace raw data while remaining broadly useful. We present a systematic analysis of embedding design in GeoFM-based EO workflows. Leveraging NeuCo-Bench, we study how backbone architecture, pretraining strategy, representation depth, spatial aggregation, and representation combination influence EO task performance. We demonstrate the usability of GeoFM embeddings by aggregating them into fixed-size representations more than 500x smaller than the raw input data. Across models, we find consistent trends: transformer backbones with mean pooling provide strong default embeddings, intermediate ResNet layers can outperform final layers, self-supervised objectives exhibit task-specific strengths, and combining embeddings from different objectives often improves robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。