让卫星图像能被自然语言理解，助力环境监测与城市规划。
From Pixels to Prose: Advancing Multi-Modal Language Models for Remote Sensing
- 用双编码器和Transformer融合遥感图像与文本信息
- 支持场景描述、变化检测等应用，提升灾害响应效率
- 适合遥感、AI、地理信息领域研究者参考
遥感已从简单的图像获取发展为整合处理视觉与文本数据的复杂系统。本文综述多模态语言模型（MLLMs）在遥感中的发展与应用，重点探讨其利用自然语言解析和描述卫星影像的能力。涵盖MLLMs的技术基础，包括双编码器架构、Transformer模型、自监督与对比学习、跨模态融合。分析遥感数据特有的挑战——空间分辨率差异、光谱丰富性及时间变化对模型性能的影响。讨论关键应用如场景描述、目标检测、变化检测、文本到图像检索、图像到文本生成及视觉问答，展示其在环境监测、城市规划和灾害响应中的价值。回顾支撑模型训练与评估的重要数据集与资源。指出计算需求、可扩展性、数据质量与领域适配等挑战。最后提出未来研究方向与技术改进路径，以进一步提升MLLM在遥感中的实用性。
原文摘要 · Abstract (English)
Remote sensing has evolved from simple image acquisition to complex systems capable of integrating and processing visual and textual data. This review examines the development and application of multi-modal language models (MLLMs) in remote sensing, focusing on their ability to interpret and describe satellite imagery using natural language. We cover the technical underpinnings of MLLMs, including dual-encoder architectures, Transformer models, self-supervised and contrastive learning, and cross-modal integration. The unique challenges of remote sensing data--varying spatial resolutions, spectral richness, and temporal changes--are analyzed for their impact on MLLM performance. Key applications such as scene description, object detection, change detection, text-to-image retrieval, image-to-text generation, and visual question answering are discussed to demonstrate their relevance in environmental monitoring, urban planning, and disaster response. We review significant datasets and resources supporting the training and evaluation of these models. Challenges related to computational demands, scalability, data quality, and domain adaptation are highlighted. We conclude by proposing future research directions and technological advancements to further enhance MLLM utility in remote sensing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。