arXiv:2605.04590cs.CVcs.AI2026-05中稿 · ICMR 2026

用修正流模型直接生成分割图,零样本效果更优。

From Diffusion to Rectified Flow: Rethinking Text-Based Segmentation

论文配图:From Diffusion to Rectified Flow: Rethinking Text-Based Segmentation
图 1 · 摘自论文原文
  • 用修正流替代扩散模型,直接学习图像到分割图的映射。
  • 单次推理即可达高精度,零样本性能显著优于扩散模型方法。
  • 无需修改结构,可将生成模型转为分割模型,适合迁移应用。

文本驱动图像分割旨在根据文本提示精确勾勒图像中物体的边界,相比传统固定类别分割更具灵活性和应用广度。近期研究发现,扩散模型(如Stable Diffusion)能提供丰富的多模态语义特征,被用于作为分割任务的特征提取器。然而,这类方法继承了扩散模型的生成特性,不利于判别性分割任务。为此,我们提出RLFSeg框架,利用修正流在潜在空间中学习从图像到分割掩码的直接映射。该方法摆脱了噪声-去噪过程及扩散模型时间步优化需求,显著提升性能,尤其在零样本场景下表现优异。通过引入标签精炼和自适应单步采样策略,模型在单次推理下也能实现更高精度。该框架无需修改预训练生成模型结构,即可将其转向判别性分割任务,展现出巨大应用潜力与研究价值。

原文摘要 · Abstract (English)

Text-based image segmentation aims to delineate object boundaries within an image from text prompts, offering higher flexibility and broader application scope compared to traditional fixed-category segmentation tasks. Recent studies have shown that diffusion models (e.g., Stable Diffusion) can provide rich multimodal semantic features, leading to studies of using diffusion models as feature extractors for segmentation tasks. Such methods, however, inherit the generative natures of diffusion models that are harmful to discriminative segmentation tasks. In response, we propose RLFSeg, a novel framework that leverages Rectified Flow to learn direct mapping from the image to the segmentation mask within the latent space. The model is thus freed from the noise-denoise process and the need to optimize the time step of diffusion models, resulting in substantially better performance than previous diffusion-based methods, especially on zero-shot scenarios. By introducing label refinement and an Adaptive One-Step Sampling strategy, the model achieves higher accuracy even on a single inference step. The framework redirects a pretrained generative model to the discriminative segmentation task with zero modification to model structure, thus reveals promising application potential and significant research value.

图像分割修正流文本引导零样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。