用冻结的扩散模型提取特征，实现海洋浮游生物细粒度分类
Diffusion Representations for Fine-Grained Image Classification: A Marine Plankton Case Study
- 从扩散过程各层和时间步中提取特征，线性分类器逐对训练
- 在平衡与长尾数据下均优于自监督方法，接近有监督性能
- 跨时空数据集仍保持高准确率，适合真实环境长期监测
扩散模型作为图像生成的前沿方法，其作为通用特征编码器的潜力尚未被充分挖掘。这些无需标签即可训练的模型通过去噪与生成过程，可视为自监督学习者，能捕捉低层与高层结构。我们发现，冻结的扩散主干网络通过探测不同层和时间步的中间去噪特征，并为每对特征训练线性分类器，可实现强大的细粒度识别能力。我们在具有实际应用价值的浮游生物监测场景中进行评估，采用受控且可比的训练设置，对比了成熟的有监督与自监督基线方法。结果表明，冻结扩散特征在平衡与自然长尾数据设置下均表现优异，性能媲美有监督方法，超越其他自监督方法。在时间与地理分布发生显著偏移的分布外数据集上，冻结扩散特征仍保持高准确率与宏平均F1分数。
原文摘要 · Abstract (English)
Diffusion models have emerged as state-of-the-art generative methods for image synthesis, yet their potential as general-purpose feature encoders remains underexplored. Trained for denoising and generation without labels, they can be interpreted as self-supervised learners that capture both low- and high-level structure. We show that a frozen diffusion backbone enables strong fine-grained recognition by probing intermediate denoising features across layers and timesteps and training a linear classifier for each pair. We evaluate this in a real-world plankton-monitoring setting with practical impact, using controlled and comparable training setups against established supervised and self-supervised baselines. Frozen diffusion features are competitive with supervised baselines and outperform other self-supervised methods in both balanced and naturally long-tailed settings. Out-of-distribution evaluations on temporally and geographically shifted plankton datasets further show that frozen diffusion features maintain strong accuracy and Macro F1 under substantial distribution shift.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。