在数据分布变化下，让模型自动发现新类别并准确分类。
Generalized Category Discovery under Domain Shifts: From Vision to Vision-Language Models
- 分离领域与语义特征，用混合增强和课程采样提升鲁棒性。
- 在多种真实域偏移场景中，相比基线提升显著，最高达12.3%准确率。
- 适配视觉与视觉语言模型，适合不同部署需求的系统集成。
广义类别发现（GCD）旨在通过已知类别的标注数据，对未知类别和已知类别的未标注样本进行分类。现有方法假设所有数据来自单一域，但现实中的未标注数据常伴随领域偏移与语义偏移。本文研究在领域偏移下的GCD问题，提出三种适配基础模型的框架：(i) HiLo 通过多级特征提取与互信息最小化，解耦领域与语义特征，并结合PatchMix增强与课程采样；(ii) HLPrompt 在HiLo基础上引入语义感知的空间提示调优，抑制背景与领域噪声；(iii) VLPrompt 利用视觉语言模型，采用因子化文本提示与跨模态一致性正则化。三者共享核心设计原则，适用于不同基础模型。在合成扰动与真实多域偏移数据集上的实验表明，该方法持续优于强基线，在多个设置下提升最高达12.3%。项目页面：https://visual-ai.github.io/hilo/
原文摘要 · Abstract (English)
Generalized Category Discovery (GCD) aims to categorize unlabelled instances from both known and unknown classes by transferring knowledge from labelled data of known classes. Existing methods assume all data comes from a single domain, yet real-world unlabelled data often exhibits domain shifts alongside semantic shifts. We study GCD under domain shifts and propose three frameworks that adapt foundation models, ranging from self-supervised vision models to vision-language models. (i) HiLo disentangles domain and semantic features through multi-level feature extraction and mutual information minimization, combined with PatchMix augmentation and curriculum sampling. (ii) HLPrompt extends HiLo with semantic-aware spatial prompt tuning to suppress background and domain noise. (iii) VLPrompt leverages vision-language models via factorized textual prompts and cross-modal consistency regularization. The three methods share core design principles while operating on different foundation backbones, making them suitable for different deployment scenarios. Extensive experiments on synthetic corruptions and real-world multi-domain shifts demonstrate consistent improvements over strong baselines. Project page: https://visual-ai.github.io/hilo/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。