arXiv:2602.12401cs.CV2026-02

提出ZeroDiff++,用扩散模型增强零样本学习中的视觉语义关联性。

ZeroDiff++: Substantial Unseen Visual-semantic Correlation in Zero-shot Learning

  • 基于扩散模型生成多样噪声样本并结合对比学习提升语义表征
  • 在三个基准上显著优于现有方法,数据稀缺下仍保持鲁棒性能
  • 适合研究零样本学习与生成模型融合的学者参考

零样本学习(ZSL)使分类器能够识别训练中未见的类别,通常采用两阶段生成方法:(1) 从已见类别中学习视觉-语义关联;(2) 从语义合成未见类别特征以训练分类器。本文发现现有生成式ZSL方法因已见类别样本稀少而产生虚假视觉-语义关联,并引入两个指标量化已见与未见类别的虚假程度。此外,指出更关键瓶颈:现有无适应性的全噪声生成器产生的特征与真实测试样本脱节,加剧虚假关联。为增强已见与未见类别的视觉-语义关联,提出基于扩散模型的ZeroDiff++框架。训练时,使用(1)扩散增强生成多样化噪声样本,(2)监督对比学习获取实例级语义表示,(3)多视图判别器结合Wasserstein互学习评估生成特征。推理时,引入(4)基于扩散的测试时自适应(DiffTTA),通过伪标签重建优化生成器;(5)基于扩散的测试时生成(DiffGen),追踪扩散去噪路径生成部分合成特征,连接真实与生成数据,缓解数据稀缺问题。在三个ZSL基准上的大量实验表明,ZeroDiff++不仅显著优于现有方法,且在训练数据稀少时仍表现稳健。代码将公开。

原文摘要 · Abstract (English)

Zero-shot Learning (ZSL) enables classifiers to recognize classes unseen during training, commonly via generative two stage methods: (1) learn visual semantic correlations from seen classes; (2) synthesize unseen class features from semantics to train classifiers. In this paper, we identify spurious visual semantic correlations in existing generative ZSL worsened by scarce seen class samples and introduce two metrics to quantify spuriousness for seen and unseen classes. Furthermore, we point out a more critical bottleneck: existing unadaptive fully noised generators produce features disconnected from real test samples, which also leads to the spurious correlation. To enhance the visual-semantic correlations on both seen and unseen classes, we propose ZeroDiff++, a diffusion-based generative framework. In training, ZeroDiff++ uses (i) diffusion augmentation to produce diverse noised samples, (ii) supervised contrastive (SC) representations for instance level semantics, and (iii) multi view discriminators with Wasserstein mutual learning to assess generated features. At generation time, we introduce (iv) Diffusion-based Test time Adaptation (DiffTTA) to adapt the generator using pseudo label reconstruction, and (v) Diffusion-based Test time Generation (DiffGen) to trace the diffusion denoising path and produce partially synthesized features that connect real and generated data, and mitigates data scarcity further. Extensive experiments on three ZSL benchmarks demonstrate that ZeroDiff++ not only achieves significant improvements over existing ZSL methods but also maintains robust performance even with scarce training data. Code would be available.

零样本学习扩散模型生成对抗语义关联

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。