arXiv:2601.14111cs.CV2026-01

用图文描述增强少样本学习,提升分类准确率

PMCE: Probabilistic Multi-Granularity Semantics with Caption-Guided Enhancement for Few-Shot Learning

  • 构建多粒度语义知识库,融合类别名与视觉统计信息
  • 通过图像描述优化支持集原型和查询特征,1次学习下提升7.71%准确率
  • 适合做少样本图像分类的科研人员和工程师参考

少样本学习旨在仅用少量标注样本识别新类别,但稀疏数据导致原型偏差大、泛化差。现有语义方法多仅在支持集应用粗粒度类别信息,未改进查询表示。本文提出PMCE框架,利用基于标题的多粒度语义增强:构建非参数化知识库,存储每个类别的视觉统计及基类的CLIP编码类别名嵌入。元测试时,根据类别名嵌入相似性检索最相关基类,聚合其统计信息生成类别先验,并通过简单最大后验(MAP)更新支持集原型。同时,冻结的BLIP图像描述器提供无标签实例描述,轻量级增强器在基类上训练,通过一致性正则化稳定噪声描述,在归纳协议下优化支持原型与查询特征。在四个基准测试上,PMCE持续优于强基线,在MiniImageNet的1次学习设置下,比最强语义对手高出7.71%绝对准确率。

原文摘要 · Abstract (English)

Few-shot learning aims to identify novel categories from only a handful of labeled samples, where prototypes estimated from scarce data are often biased and generalize poorly. Semantic-based methods alleviate this by introducing coarse class-level information, but they are mostly applied on the support side, leaving query representations unchanged. In this paper, we present PMCE, a Probabilistic few-shot framework that leverages Multi-granularity semantics with Caption-guided Enhancement. PMCE constructs a nonparametric knowledge bank that stores visual statistics for each category as well as CLIP-encoded class name embeddings of the base classes. At meta-test time, the most relevant base classes are retrieved based on the similarities of class name embeddings for each novel category. These statistics are then aggregated into category-specific prior information and fused with the support set prototypes via a simple MAP update. Simultaneously, a frozen BLIP captioner provides label-free instance-level image descriptions, and a lightweight enhancer trained on base classes optimizes both support prototypes and query features under an inductive protocol with a consistency regularization to stabilize noisy captions. Experiments on four benchmarks show that PMCE consistently improves over strong baselines, achieving up to 7.71% absolute gain over the strongest semantic competitor on MiniImageNet in the 1-shot setting. Our code is available at https://anonymous.4open.science/r/PMCE-275D

少样本学习图文对齐语义增强

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。