arXiv:2507.20511cs.CV2025-07

用大模型挖掘图像关键属性,提升少样本分类能力

Beyond Class Tokens: LLM-guided Dominant Property Mining for Few-shot Classification

  • 通过大模型生成多视觉属性标记,超越传统类别词嵌入
  • 在11个数据集上实现更优少样本分类效果
  • 适合关注少样本学习与视觉属性建模的研究者

少样本学习(FSL)旨在仅用少量图像识别新类别,但数据稀缺导致泛化困难。现有基于CLIP的方法虽借助类别名称文本表示缓解问题,但简单对齐视觉与文本嵌入会削弱新类别间的视觉区分性。为此,我们提出BCT-CLIP方法,通过对比学习挖掘超越类别词的主导属性。利用大语言模型先验知识,构建包含全局类别表征与局部块感知属性嵌入的综合结构表示。具体包括:带块感知交叉注意力的多属性生成器(MPG)、基于聚类剪枝的大语言模型辅助检索流程,以及面向属性标记的新型对比学习策略。在11个广泛使用的数据集上验证表明,该方法显著提升了判别性类别特定表示学习与少样本分类性能。

原文摘要 · Abstract (English)

Few-shot Learning (FSL), which endeavors to develop the generalization ability for recognizing novel classes using only a few images, faces significant challenges due to data scarcity. Recent CLIP-like methods based on contrastive language-image pertaining mitigate the issue by leveraging textual representation of the class name for unseen image discovery. Despite the achieved success, simply aligning visual representations to class name embeddings would compromise the visual diversity for novel class discrimination. To this end, we proposed a novel Few-Shot Learning (FSL) method (BCT-CLIP) that explores \textbf{dominating properties} via contrastive learning beyond simply using class tokens. Through leveraging LLM-based prior knowledge, our method pushes forward FSL with comprehensive structural image representations, including both global category representation and the patch-aware property embeddings. In particular, we presented a novel multi-property generator (MPG) with patch-aware cross-attentions to generate multiple visual property tokens, a Large-Language Model (LLM)-assistant retrieval procedure with clustering-based pruning to obtain dominating property descriptions, and a new contrastive learning strategy for property-token learning. The superior performances on the 11 widely used datasets demonstrate that our investigation of dominating properties advances discriminative class-specific representation learning and few-shot classification.

少样本学习视觉属性大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。