用区域选择替代坐标生成,提升遥感图像变化定位精度
From Coordinates to Candidate Regions: Temporal Change Localization via Region Selection in Remote Sensing Multimodal LLMs

- 以候选区域代替坐标作为文本条件下的输入序列
- 在多时相遥感图像中变化定位准确率显著超越基线方法
- 适合需要精确定位变化区域的遥感分析人员
遥感多模态大语言模型(RS-MLLMs)虽已提升卫星影像场景理解与视觉问答能力,但精确定位特定目标或变化区域仍具挑战。现有方法依赖生成边界框坐标作为标记序列,对遥感中细小密集目标敏感,且在需同时定位多个目标时误差增大。本文提出一种针对遥感场景的区域选择范式,扩展至多图像序列的时间变化定位任务。框架采用文本条件下的区域提议模块,将每个候选区域编码为携带帧内视觉特征并融合空间与时间线索的特殊标记,由大语言模型通过选择区域标记完成定位。构建覆盖单图与多时相设置的多任务训练与评估体系,涵盖定位、指代表达、视觉定位与理解任务。实验表明,该方法在时间变化定位上显著优于坐标生成基线,同时提升单图视觉定位性能,并保持良好的理解能力。基于最优分析分解区域提议器与语言模型选择器的贡献,提供该框架独有的诊断洞察。代码将开源于 https://github.com/juwan-kr/RS-RegionSelect。
原文摘要 · Abstract (English)
Remote sensing multimodal large language models (RS-MLLMs) have advanced scene understanding and visual question answering over satellite imagery, yet localizing specific objects or changed regions remains challenging. Existing approaches rely on generating bounding box coordinates as token sequences, which is fragile for the small, densely packed objects common in remote sensing and increasingly error-prone when multiple targets must be localized simultaneously. In this work, we present an RS-specific formulation of the region selection paradigm, previously explored in natural-image MLLMs, and extend it to temporal change localization over multi-image sequences. Our framework employs a text-conditioned region proposal module, encodes each candidate as special tokens carrying per-frame visual features enriched with spatial and temporal cues, and lets the LLM localize targets by selecting region tokens in its response. We construct a multi-task training and evaluation suite spanning localization, referring expression, visual grounding, and understanding tasks across single-image and multi-temporal settings. Experiments show that our approach substantially outperforms coordinate-generation baselines on temporal change localization, while improving single-image visual grounding and maintaining competitive understanding performance. Oracle analysis decomposes the contributions of the region proposer and the LLM selector, providing diagnostic insight unique to this framework. Our code will be available at https://github.com/juwan-kr/RS-RegionSelect.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。