用自然语言提取可解释特征，让模型少样本识别更准
Verbalized Representation Learning for Interpretable Few-Shot Generalization
- 用视觉语言模型提取类别间差异和类内共性，转为自然语言特征
- 仅用5%数据就比顶尖方法高24%准确率，模型更小
- 学出的特征比人工标注还强20%，适合需要可解释性的场景
人类仅需观察少量样本就能识别物体，这种能力源于对现实世界的语言理解。构建可解释的语义表征能显著提升低数据场景下的模型泛化能力。本文提出口语化表征学习（VRL），通过视觉语言模型自动从少样本数据中提取人类可理解的特征用于物体识别。该方法利用视觉语言模型识别不同类别间的判别性特征和同一类别内的共有特性，并将这些语义特征转化为数值向量。实验表明，在相同模型规模下，VRL相比现有最优方法在少样本设置中准确率提升24%，且仅需1/20的数据量和更小的模型。此外，相较于人工标注属性，VRL学到的特征在下游分类任务中表现高出20%。代码已开源。
原文摘要 · Abstract (English)
Humans recognize objects after observing only a few examples, a remarkable capability enabled by their inherent language understanding of the real-world environment. Developing verbalized and interpretable representation can significantly improve model generalization in low-data settings. In this work, we propose Verbalized Representation Learning (VRL), a novel approach for automatically extracting human-interpretable features for object recognition using few-shot data. Our method uniquely captures inter-class differences and intra-class commonalities in the form of natural language by employing a Vision-Language Model (VLM) to identify key discriminative features between different classes and shared characteristics within the same class. These verbalized features are then mapped to numeric vectors through the VLM. The resulting feature vectors can be further utilized to train and infer with downstream classifiers. Experimental results show that, at the same model scale, VRL achieves a 24% absolute improvement over prior state-of-the-art methods while using 95% less data and a smaller mode. Furthermore, compared to human-labeled attributes, the features learned by VRL exhibit a 20% absolute gain when used for downstream classification tasks. Code is available at: https://github.com/joeyy5588/VRL/tree/main.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。