通过提示学习提升遥感图像实例分割精度,支持框提示且推理仅需40毫秒
Insight Any Instance: Promptable Instance Segmentation for Remote Sensing Images
- 设计局部与全局到局部提示模块,增强小目标和不均衡场景下的特征表达
- 引入提案面积损失函数,解耦尺度维度,显著提升小实例分割性能
- 可快速响应框提示,适合需要交互式分割的遥感应用
遥感图像实例分割在土地规划、智能交通等领域至关重要,但受限于前景与背景比例失衡及实例尺寸有限。现有模型多依赖深层特征学习并包含多次下采样操作,不利于遥感图像中细小实例的分割,导致性能受限。受提示学习在视觉任务中优异表现启发,本文提出一种新提示范式:首先设计局部提示模块,从原始局部令牌中挖掘特定实例的提示信息;其次设计全局到局部提示模块,将全局上下文信息建模至实例所在局部令牌;最后设计提案面积损失函数,为提案增加解耦尺度维度,以更好发挥上述两个模块潜力。所提方法可将现有实例分割模型扩展为可提示的实例分割模型,支持基于指定框的实例分割。单次提示分割耗时仅40毫秒。在四个遥感图像实例分割数据集上对多个现有模型进行评估,实验充分证明该方法有效缓解前述问题,是遥感图像实例分割的有力候选方案。
原文摘要 · Abstract (English)
Instance segmentation of remote sensing images (RSIs) is an essential task for a wide range of applications such as land planning and intelligent transport. Instance segmentation of RSIs is constantly plagued by the unbalanced ratio of foreground and background and limited instance size. And most of the instance segmentation models are based on deep feature learning and contain operations such as multiple downsampling, which is harmful to instance segmentation of RSIs, and thus the performance is still limited. Inspired by the recent superior performance of prompt learning in visual tasks, we propose a new prompt paradigm to address the above issues. Based on the existing instance segmentation model, firstly, a local prompt module is designed to mine local prompt information from original local tokens for specific instances; secondly, a global-to-local prompt module is designed to model the contextual information from the global tokens to the local tokens where the instances are located for specific instances. Finally, a proposal's area loss function is designed to add a decoupling dimension for proposals on the scale to better exploit the potential of the above two prompt modules. It is worth mentioning that our proposed approach can extend the instance segmentation model to a promptable instance segmentation model, i.e., to segment the instances with the specific boxes prompt. The time consumption for each promptable instance segmentation process is only 40 ms. The paper evaluates the effectiveness of our proposed approach based on several existing models in four instance segmentation datasets of RSIs, and thorough experiments prove that our proposed approach is effective for addressing the above issues and is a competitive model for instance segmentation of RSIs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。