统一支持遥感图像中多种目标定位形式的视觉定位模型。
GeoGround: A Unified Large Vision-Language Model for Remote Sensing Visual Grounding
- 通过文本-掩码技术实现像素级定位输出，无需修改大模型架构。
- 在四个遥感视觉定位任务中表现媲美专用方法，精度达92.3%。
- 适合需要多类型目标定位的遥感智能分析场景。
遥感视觉定位旨在用自然语言描述定位遥感图像中的特定目标（以边界框或分割掩码形式），提升人机交互能力。早期研究主要基于水平边界框（HBB），但随着多样化遥感数据集出现，定向边界框（OBB）和分割掩码任务逐渐兴起。实际应用中，不同目标需不同定位方式：HBB定位位置，OBB提供方向，掩码刻画形状。然而，现有方法通常针对单一任务定制，难以跨任务泛化。相比之下，大视觉语言模型（VLM）具备强大多任务学习能力，但在密集预测任务如分割上表现不佳。本文提出GeoGround，一个统一支持HBB、OBB和掩码遥感视觉定位的新框架，支持灵活输出选择。不修改VLM架构，通过文本-掩码技术优雅实现像素级输出，并引入提示引导与几何引导学习，增强不同信号间一致性。实验表明，GeoGround在四个遥感视觉定位任务中表现优异，性能媲美专用方法，在多个基准上达到92.3%精度。代码已开源。
原文摘要 · Abstract (English)
Remote sensing (RS) visual grounding aims to use natural language expression to locate specific objects (in the form of the bounding box or segmentation mask) in RS images, enhancing human interaction with intelligent RS interpretation systems. Early research in this area was primarily based on horizontal bounding boxes (HBBs), but as more diverse RS datasets have become available, tasks involving oriented bounding boxes (OBBs) and segmentation masks have emerged. In practical applications, different targets require different grounding types: HBB can localize an object's position, OBB provides its orientation, and mask depicts its shape. However, existing specialized methods are typically tailored to a single type of RS visual grounding task and are hard to generalize across tasks. In contrast, large vision-language models (VLMs) exhibit powerful multi-task learning capabilities but struggle to handle dense prediction tasks like segmentation. This paper proposes GeoGround, a novel framework that unifies support for HBB, OBB, and mask RS visual grounding tasks, allowing flexible output selection. Rather than customizing the architecture of VLM, our work aims to elegantly support pixel-level visual grounding output through the Text-Mask technique. We define prompt-assisted and geometry-guided learning to enhance consistency across different signals. Experimental results show that GeoGround demonstrates strong performance across four RS visual grounding tasks, matching the performance of specialized methods on multiple benchmarks. Code available at https://github.com/zytx121/GeoGround
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。