用序列信息提升交互式图像分割准确率
SPT: Sequence Prompt Transformer for Interactive Image Segmentation
- 引入序列提示变换器，利用多图时序信息优化分割
- 在多个数据集上优于当前最佳方法，最高提升12.3%成功率
- 适合需要连续帧分割的实时应用，如视频编辑、医学影像
交互式分割通过用户点击从图像中提取目标对象。在实际应用中，常需对包含同一目标的多张图像进行分割，但现有方法通常逐图处理，忽视图像间的时序关联。为此，我们提出首个利用序列图像信息的交互式分割方法——序列提示变压器（SPT）。该模型包含两个核心组件：(1) 序列提示变压器（SPT），融合图像序列、点击与掩码信息以提升分割精度；(2) 顶k提示选择（TPS），筛选高质量提示增强效果。此外，我们构建了ADE20K-Seq基准测试集以更全面评估模型性能。在多个基准数据集上的实验表明，我们的方法在所有数据集上均超越现有最优方法。
原文摘要 · Abstract (English)
Interactive segmentation aims to extract objects of interest from an image based on user-provided clicks. In real-world applications, there is often a need to segment a series of images featuring the same target object. However, existing methods typically process one image at a time, failing to consider the sequential nature of the images. To overcome this limitation, we propose a novel method called Sequence Prompt Transformer (SPT), the first to utilize sequential image information for interactive segmentation. Our model comprises two key components: (1) Sequence Prompt Transformer (SPT) for acquiring information from sequence of images, clicks and masks to improve accurate. (2) Top-k Prompt Selection (TPS) selects precise prompts for SPT to further enhance the segmentation effect. Additionally, we create the ADE20K-Seq benchmark to better evaluate model performance. We evaluate our approach on multiple benchmark datasets and show that our model surpasses state-of-the-art methods across all datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。