构建百万级遥感图像指令分割数据集,实现跨场景通用语义分割。
UniGeoSeg: Towards Unified Open-World Segmentation for Geospatial Scenes
- 基于自动筛选与指令生成构建百万级遥感指令分割数据集
- 在GeoSeg-Bench上达到顶尖性能并展现强零样本泛化能力
- 适合遥感智能分析、地理信息处理方向的研究者参考
遥感中基于指令的分割通过文本引导生成掩码,具有高可访问性和通用性潜力。然而,现有方法存在任务定义碎片化和指令数据稀缺问题,制约理解与泛化能力。为此,我们提出GeoSeg-1M,首个百万级遥感指令驱动分割数据集,通过自动化掩码过滤与指令生成流程,从多个公开数据集合成指代、交互与推理类分割指令。GeoSeg-1M包含59万张图像、117个类别及110万组图像-掩码-指令三元组。在此基础上,我们进一步构建GeoSeg-Bench,一个用于评估上下文理解与推理能力的挑战性基准,涵盖多样化指令任务与复杂地理空间场景。同时,提出UniGeoSeg统一框架,作为强基线模型,融合任务感知文本增强、潜在知识记忆与渐进式训练策略,支持多任务学习。大量实验表明,UniGeoSeg在GeoSeg-Bench及多个公开基准上均达领先水平,并具备优异零样本泛化能力。数据集与源代码已开源至https://github.com/MiliLab/UniGeoSeg。
原文摘要 · Abstract (English)
Instruction-driven segmentation in remote sensing generates masks from guidance, offering great potential for accessible and generalizable applications. However, existing methods suffer from fragmented task formulations and limited instruction data, hindering effective understanding and generalization. To address these issues, we introduce GeoSeg-1M, the first million-scale dataset for remote sensing instruction-driven segmentation, constructed via an automatic mask filtering and instruction generation pipeline that synthesizes referring, interactive, and reasoning segmentation instructions from multiple public datasets. GeoSeg-1M contains 590K images, 117 categories, and 1.1M image-mask-instruction triplets. Building upon this foundation, we further curate GeoSeg-Bench, a challenging benchmark designed to evaluate contextual understanding and reasoning capabilities across diverse instruction-driven tasks and complex geospatial scenes. Furthermore, we present UniGeoSeg, a unified framework that serves as a strong baseline, incorporating task-aware text enhancement, latent knowledge memory, and a progressive training strategy to facilitate multi-task learning. Extensive experiments demonstrate the state-of-the-art performance of UniGeoSeg across GeoSeg-Bench and diverse public benchmarks, while exhibiting strong zero-shot generalization. Datasets and source code were released at https://github.com/MiliLab/UniGeoSeg.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。