用扩散模型实现无需掩码的全局图像分割，支持模糊性建模。
Towards Agnostic and Holistic Universal Image Segmentation with Bit Diffusion
- 基于扩散模型直接预测完整分割图，不依赖传统掩码框架。
- 引入二维灰度码位置感知调色板，提升分割精度。
- 可建模不确定性，适合需要鲁棒性的实际应用。
本文提出一种基于扩散模型的通用图像分割框架，实现了不依赖掩码的无偏分割，通过整体方式预测完整分割结果。针对离散设置，我们提出若干关键改进：采用二维灰度码排序的位置感知调色板显著提升性能；添加最终的tanh激活函数对离散数据至关重要；在优化扩散参数时，无论使用何种预测类型，sigmoid损失加权始终优于其他方法，因此选用x-prediction。尽管当前模型尚未超越领先的掩码基架构，但已显著缩小性能差距，并具备这些模型所缺乏的合理模糊性建模能力。所有模型均从零开始训练，我们认为结合大规模预训练或可提示条件化，未来有望达到竞争力水平。
原文摘要 · Abstract (English)
This paper introduces a diffusion-based framework for universal image segmentation, making agnostic segmentation possible without depending on mask-based frameworks and instead predicting the full segmentation in a holistic manner. We present several key adaptations to diffusion models, which are important in this discrete setting. Notably, we show that a location-aware palette with our 2D gray code ordering improves performance. Adding a final tanh activation function is crucial for discrete data. On optimizing diffusion parameters, the sigmoid loss weighting consistently outperforms alternatives, regardless of the prediction type used, and we settle on x-prediction. While our current model does not yet surpass leading mask-based architectures, it narrows the performance gap and introduces unique capabilities, such as principled ambiguity modeling, that these models lack. All models were trained from scratch, and we believe that combining our proposed improvements with large-scale pretraining or promptable conditioning could lead to competitive models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。