arXiv:2512.05025cs.CV2025-12被引 3

RAMEN统一处理多源遥感数据,可调分辨率实现高效推理。

RAMEN: Resolution-Adjustable Multimodal Encoder for Earth Observation

  • 将分辨率设为可控参数,统一编码跨模态遥感数据。
  • 在PANGAEA基准上超越更大模型,支持未知传感器配置迁移。
  • 适合需要灵活处理多分辨率遥感数据的研究与应用。

地球观测(EO)数据涵盖从高分辨率光学影像到低分辨率多光谱产品或雷达时序的广泛空间、光谱和时间分辨率。尽管近期基础模型提升了多模态融合能力,但多数模型依赖固定输入分辨率或基于特定传感器的编码器,限制了在异构EO模态间的泛化能力。为此,我们提出RAMEN——一种分辨率可调的多模态编码器,以全传感器无关方式学习共享视觉表示。RAMEN将模态及空间、时间分辨率作为关键输入特征,在统一潜在空间中实现跨模态一致分析。其核心方法是将空间分辨率定义为可调控输出参数,使用户在推理阶段直接控制细节程度,明确权衡空间精度与计算成本。我们训练单一统一的Transformer编码器,重构来自多样来源的掩码多模态EO数据,确保对传感器与分辨率的泛化能力。预训练后,RAMEN在包含多种多传感器、多分辨率下游任务的社区标准基准PANGAEA上,有效迁移到已知和未见传感器配置,并优于更大的现有模型。代码与预训练模型已开源:https://github.com/nicolashoudre/RAMEN。

原文摘要 · Abstract (English)

Earth observation (EO) data spans a wide range of spatial, spectral, and temporal resolutions, from high-resolution optical imagery to low resolution multispectral products or radar time series. While recent foundation models have improved multimodal integration for learning meaningful representations, they often expect fixed input resolutions or are based on sensor-specific encoders limiting generalization across heterogeneous EO modalities. To overcome these limitations we introduce RAMEN, a resolution-adjustable multimodal encoder that learns a shared visual representation across EO data in a fully sensor-agnostic manner. RAMEN treats the modality and spatial and temporal resolutions as key input data features, enabling coherent analysis across modalities within a unified latent space. Its main methodological contribution is to define spatial resolution as a controllable output parameter, giving users direct control over the desired level of detail at inference and allowing explicit trade-offs between spatial precision and computational cost. We train a single, unified transformer encoder reconstructing masked multimodal EO data drawn from diverse sources, ensuring generalization across sensors and resolutions. Once pretrained, RAMEN transfers effectively to both known and unseen sensor configurations and outperforms larger state-of-the-art models on the community-standard PANGAEA benchmark, containing various multi-sensor and multi-resolution downstream tasks. Our code and pretrained model are available at https://github.com/nicolashoudre/RAMEN.

遥感多模态可调分辨率Transformer

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。