用随机生成模型融合多源遥感数据，实现跨模态任意转换。
COP-GEN: Latent Diffusion Transformer for Copernicus Earth Observation Data
- 基于潜在扩散变换器建模多模态遥感数据联合分布
- 在哨兵-2数据上覆盖90%真实观测流形，反射率范围达63%
- 支持零样本跨模态生成，适合遥感数据补全与仿真任务
地球观测应用日益依赖光学、雷达、高程和土地覆盖等多源传感器数据。模态间关系对数据融合至关重要，但具有非单射性：相同条件可能对应多个物理上合理的观测结果，需以条件分布形式建模。确定性模型会退化为条件均值，无法表达数据补全与跨模态翻译所需的不确定性与变异性。本文提出COP-GEN，一种多模态潜在扩散变换器，在原始空间分辨率下建模异构地球观测模态的联合分布。通过将跨模态映射参数化为条件分布，COP-GEN实现灵活的任意模态条件生成，包括无需任务特定训练的零样本模态转换。实验表明，COP-GEN生成多样且物理一致的结果，同时在光学、雷达和高程模态上保持强峰值保真度。定性和定量分析显示，模型捕捉到有意义的跨模态结构，并随条件信息增加自适应调整输出不确定性。我们发布了基于多时相哨兵-2观测的随机基准，支持生成式遥感模型的分布级比较。在该基准上，COP-GEN覆盖90%的真实观测流形和63%的每波段反射率范围，而最强对比方法分别仅覆盖2.8%和18%。这些结果凸显了随机生成建模在地球观测中的重要性，并推动超越单参考点对点度量的评估范式。
原文摘要 · Abstract (English)
Earth observation applications increasingly rely on data from multiple sensors, including optical, radar, elevation, and land-cover. Relationships between modalities are fundamental for data integration but are inherently non-injective: identical conditioning information can correspond to multiple physically plausible observations, and should be parametrised as conditional distributions. Deterministic models, by contrast, collapse toward conditional means and fail to represent the uncertainty and variability required for tasks such as data completion and cross-sensor translation. We introduce COP-GEN, a multimodal latent diffusion transformer that models the joint distribution of heterogeneous EO modalities at their native spatial resolutions. By parameterising cross-modal mappings as conditional distributions, COP-GEN enables flexible any-to-any conditional generation, including zero-shot modality translation without task-specific retraining. Experiments show that COP-GEN generates diverse yet physically consistent realisations while maintaining strong peak fidelity across optical, radar, and elevation modalities. Qualitative and quantitative analyses demonstrate that the model captures meaningful cross-modal structure and adapts its output uncertainty as conditioning information increases. We release a stochastic benchmark built from multi-temporal Sentinel-2 observations that enables distribution-level comparison of generative EO models. On this benchmark, COP-GEN covers 90% of the real observation manifold and 63% of its per-band reflectance range, while the strongest competing method collapses to 2.8% and 18%, respectively. These results highlight the importance of stochastic generative modeling for EO and motivate evaluation protocols beyond single-reference, pointwise metrics. Website: https://miquel-espinosa.github.io/cop-gen
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。