用上下文示例实现简单高效图像分割，解决任务模糊问题。
A Simple Image Segmentation Framework via In-Context Examples
- 通过交互模块增强上下文信息关联性
- 采用匹配变换器与匈牙利算法消除任务差异
- 适用于多任务分割场景，提升泛化能力
近期研究探索了能在统一上下文学习框架中处理多种图像分割任务的通用分割模型。然而，这些方法在上下文分割中仍面临任务模糊问题，因为并非所有上下文示例都能准确传达任务信息。为此，我们提出 SINE，一种利用上下文示例的简单图像分割框架。该方法采用 Transformer 编码器-解码器结构，编码器提供高质量图像表征，解码器生成多个任务特定的输出掩码，有效消除任务模糊性。具体地，引入上下文交互模块以补充上下文信息并建立目标图像与上下文示例间的关联；设计匹配变换器，结合固定匹配与匈牙利算法，消除不同任务间的差异。此外，我们进一步完善了当前上下文图像分割的评估体系，旨在促进对这些模型的全面评估。在多种分割任务上的实验验证了所提方法的有效性。
原文摘要 · Abstract (English)
Recently, there have been explorations of generalist segmentation models that can effectively tackle a variety of image segmentation tasks within a unified in-context learning framework. However, these methods still struggle with task ambiguity in in-context segmentation, as not all in-context examples can accurately convey the task information. In order to address this issue, we present SINE, a simple image Segmentation framework utilizing in-context examples. Our approach leverages a Transformer encoder-decoder structure, where the encoder provides high-quality image representations, and the decoder is designed to yield multiple task-specific output masks to effectively eliminate task ambiguity. Specifically, we introduce an In-context Interaction module to complement in-context information and produce correlations between the target image and the in-context example and a Matching Transformer that uses fixed matching and a Hungarian algorithm to eliminate differences between different tasks. In addition, we have further perfected the current evaluation system for in-context image segmentation, aiming to facilitate a holistic appraisal of these models. Experiments on various segmentation tasks show the effectiveness of the proposed method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。