arXiv:2410.10105cs.CV2024-10ICLR被引 9

用扩散模型生成高精度二值分割图,速度快且细节清晰。

High-Precision Dichotomous Image Segmentation via Probing Diffusion Capacity

  • 利用预训练扩散模型的U-Net提取图像特征,结合单步去噪加速推理。
  • 在DIS5K数据集上达到当前最优性能,生成边缘更锐利的二值掩码。
  • 适合需要快速生成高分辨率精细分割结果的场景,如医学影像分析。

在高分辨率、细粒度图像分割领域,核心挑战在于平衡广泛的上下文感知与精确的细节刻画能力。扩散模型(如SD V2.1)基于数十亿图像-文本对训练,具备卓越的图像合成质量、精细细节还原能力和强上下文理解力,为高分辨率分割提供了新思路。本文提出DiffDIS,一种基于扩散模型的分割框架,通过挖掘预训练U-Net的表征能力,并采用任务特定的稳定单步去噪策略,在显著降低推理时间的同时保持高质量生成。此外,引入辅助边缘生成任务,增强边界细节保留,并缓解扩散模型的概率性输出与分割任务确定性需求之间的矛盾。实验表明,DiffDIS在DIS5K数据集上实现最先进的分割性能,具备高效的推理速度。代码将公开于https://github.com/qianyu-dlut/DiffDIS。

原文摘要 · Abstract (English)

In the realm of high-resolution (HR), fine-grained image segmentation, the primary challenge is balancing broad contextual awareness with the precision required for detailed object delineation, capturing intricate details and the finest edges of objects. Diffusion models, trained on vast datasets comprising billions of image-text pairs, such as SD V2.1, have revolutionized text-to-image synthesis by delivering exceptional quality, fine detail resolution, and strong contextual awareness, making them an attractive solution for high-resolution image segmentation. To this end, we propose DiffDIS, a diffusion-driven segmentation model that taps into the potential of the pre-trained U-Net within diffusion models, specifically designed for high-resolution, fine-grained object segmentation. By leveraging the robust generalization capabilities and rich, versatile image representation prior of the SD models, coupled with a task-specific stable one-step denoising approach, we significantly reduce the inference time while preserving high-fidelity, detailed generation. Additionally, we introduce an auxiliary edge generation task to not only enhance the preservation of fine details of the object boundaries, but reconcile the probabilistic nature of diffusion with the deterministic demands of segmentation. With these refined strategies in place, DiffDIS serves as a rapid object mask generation model, specifically optimized for generating detailed binary maps at high resolutions, while demonstrating impressive accuracy and swift processing. Experiments on the DIS5K dataset demonstrate the superiority of DiffDIS, achieving state-of-the-art results through a streamlined inference process. The source code will be publicly available at https://github.com/qianyu-dlut/DiffDIS.

图像分割扩散模型高精度边缘生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。