首个统一卫星时序图变化理解与预测的框架
TAMMs: Change Understanding and Forecasting in Satellite Image Time Series with Temporal-Aware Multimodal Models
- 用时序自适应模块增强大模型对长期动态的理解
- 在变化理解任务上超越现有方法，预测更连贯
- 适合遥感、气候监测等需要时序分析的研究者
卫星图像时序分析中的时序变化描述（TCD）与未来图像预测（FSIF）长期分离，均受限于长程时间动态建模。为此，我们提出TAMMs——首个基于多模态大语言模型-扩散架构的统一框架，可同时完成两项任务。TAMMs引入两个核心创新：时序自适应模块（TAM）提升冻结大模型对长程动态的理解能力；语义融合控制注入（SFCI）机制将变化认知转化为细粒度生成控制信号。该协同设计使TCD任务的感知直接优化FSIF的一致性。大量实验表明，TAMMs在两项任务上均显著优于当前最优专用基线。数据集见 https://huggingface.co/datasets/IceInPot/TAMMs。
原文摘要 · Abstract (English)
Temporal Change Description (TCD) and Future Satellite Image Forecasting (FSIF) are critical, yet historically disjointed tasks in Satellite Image Time Series (SITS) analysis. Both are fundamentally limited by the common challenge of modeling long-range temporal dynamics. To explore how to improve the performance of methods on both tasks simultaneously by enhancing long-range temporal understanding capabilities, we introduce **TAMMs**, the first unified framework designed to jointly perform TCD and FSIF within a single MLLM-diffusion architecture. TAMMs introduces two key innovations: Temporal Adaptation Modules (**TAM**) enhance frozen MLLM's ability to comprehend long-range dynamics, and Semantic-Fused Control Injection (**SFCI**) mechanism translates this change understanding into fine-grained generative control. This synergistic design makes the understanding from the TCD task to directly inform and improve the consistency of the FSIF task. Extensive experiments demonstrate TAMMs significantly outperforms state-of-the-art specialist baselines on both tasks. Our dataset can be found at https://huggingface.co/datasets/IceInPot/TAMMs .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。