针对遥感图像交互分割，提出高效适配视觉大模型的新框架。
VFM-ISRefiner: Towards Better Adapting Vision Foundation Models for Interactive Segmentation of Remote Sensing Images
- 用适配器策略保留大模型通用性，专注学习遥感特有的空间与边界特征。
- 融合卷积与Transformer的混合注意力机制，提升对尺度变化和复杂背景的鲁棒性。
- 通过历史点击信息优化概率图，减少交互次数，适合实际遥感标注场景。
交互式图像分割（IIS）在生成遥感影像精确标注中至关重要，因目标常具尺度差异、不规则边界和复杂背景。现有方法多针对自然图像设计，难以泛化至遥感领域，受限于标注数据少与计算开销高。为此，我们提出专用于遥感图像的点击式交互分割框架 RS-ISRefiner。该框架采用基于适配器的调优策略，在保持视觉基础模型通用表征的同时，高效学习遥感特有空间与边界特性。通过融合卷积局部建模与Transformer全局推理的混合注意力机制，增强对尺度多样性与场景复杂性的鲁棒性。此外，改进的概率图调制方案有效融合历史用户交互信息，实现更稳定迭代优化与更高边界精度。在六组遥感数据集（iSAID、ISPRS Potsdam、SandBar、NWPU、LoveDA Urban、WHUBuilding）上的实验表明，RS-ISRefiner 在分割精度、效率与交互成本上均持续优于当前最优 IIS 方法。结果验证了框架的有效性与泛化能力，适用于高精度实例分割的实际遥感应用。代码已开源：https://github.com/wondelyan/VFM-ISRefiner。
原文摘要 · Abstract (English)
Interactive image segmentation(IIS) plays a critical role in generating precise annotations for remote sensing imagery, where objects often exhibit scale variations, irregular boundaries and complex backgrounds. However, existing IIS methods, primarily designed for natural images, struggle to generalize to remote sensing domains due to limited annotated data and computational overhead. To address these challenges, we proposed RS-ISRefiner, a novel click-based IIS framework tailored for remote sensing images. The framework employs an adapter-based tuning strategy that preserves the general representations of Vision Foundation Models while enabling efficient learning of remote sensing-specific spatial and boundary characteristics. A hybrid attention mechanism integrating convolutional local modeling with Transformer-based global reasoning enhances robustness against scale diversity and scene complexity. Furthermore, an improved probability map modulation scheme effectively incorporates historical user interactions, yielding more stable iterative refinement and higher boundary accuracy. Comprehensive experiments on six remote sensing datasets, including iSAID, ISPRS Potsdam, SandBar, NWPU, LoveDA Urban and WHUBuilding, demonstrate that RS-ISRefiner consistently outperforms state-of-the-art IIS methods in terms of segmentation accuracy, efficiency and interaction cost. These results confirm the effectiveness and generalizability of our framework, making it highly suitable for high-quality instance segmentation in practical remote sensing scenarios. The codes are available at https://github.com/wondelyan/VFM-ISRefiner .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。