用扩散模型做图像分类预训练，无需大量标注数据
Image Classification Using a Diffusion Model as a Pre-Training Model
- 用ViT特征控制扩散过程，实现条件生成
- 脑出血检测零样本分类准确率提升6.15%，F1提升13.60%
- 适合无标注数据场景下的医学图像分类
本文提出一种融合表示条件机制的扩散模型，利用视觉变换器（ViT）提取的特征来调控基于Transformer的扩散模型内部过程，实现表示条件化的数据生成。该方法通过在无标签数据上进行自监督学习，缓解了大规模标注数据的需求难题。我们在脑部影像血肿检测的零样本分类任务中评估该方法，结果表明，相较于强基线DINOv2，本方法在准确率上提升6.15%，F1分数提升13.60%，充分验证了其在图像分类中的有效性。
原文摘要 · Abstract (English)
In this paper, we propose a diffusion model that integrates a representation-conditioning mechanism, where the representations derived from a Vision Transformer (ViT) are used to condition the internal process of a Transformer-based diffusion model. This approach enables representation-conditioned data generation, addressing the challenge of requiring large-scale labeled datasets by leveraging self-supervised learning on unlabeled data. We evaluate our method through a zero-shot classification task for hematoma detection in brain imaging. Compared to the strong contrastive learning baseline, DINOv2, our method achieves a notable improvement of +6.15% in accuracy and +13.60% in F1-score, demonstrating its effectiveness in image classification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。