对比开放词汇与闭集检测,发现难描述物体用少样本检测更该选闭集方法
Open-vocabulary vs. Closed-set: Best Practice for Few-shot Object Detection Considering Text Describability
- 用CLIP零样本分类精度量化物体文本可描述性
- 低可描述性物体上开放词汇检测效果不优于闭集方法
- 数据多样性优势在难描述类别上反而有害
开放词汇目标检测(OVD)通过仅依赖类名等语言描述来识别特定物体,无需图像样本,受到广泛关注。但在真实场景中,目标类别常难以用文字描述,只能提供图像示例,而高质量样本往往难以获取,因此少样本目标检测(FSOD)需求迫切。一个关键问题是:能否将OVD的优势延伸至难以文本描述的类别?相比仅学习预定义类别的闭集检测(COD),OVD的额外开销是否值得?为此,我们提出使用CLIP的零样本图像分类准确率来量化目标检测数据集的文本可描述性,据此对不同可描述性的数据集进行分类,并在各类别中实证评估OVD与COD方法在少样本下的性能。结果表明:一、在同等预训练条件下,低文本可描述性的类别中,OVD与COD表现差异微小;二、尽管OVD能利用比专有数据更多样化的数据,增加训练规模,但对低可描述性类别可能适得其反。这些发现为实践者在新兴OVD方法中提供了重要决策依据。
原文摘要 · Abstract (English)
Open-vocabulary object detection (OVD), detecting specific classes of objects using only their linguistic descriptions (e.g., class names) without any image samples, has garnered significant attention. However, in real-world applications, the target class concepts is often hard to describe in text and the only way to specify target objects is to provide their image examples, yet it is often challenging to obtain a good number of samples. Thus, there is a high demand from practitioners for few-shot object detection (FSOD). A natural question arises: Can the benefits of OVD extend to FSOD for object classes that are difficult to describe in text? Compared to traditional methods that learn only predefined classes (referred to in this paper as closed-set object detection, COD), can the extra cost of OVD be justified? To answer these questions, we propose a method to quantify the ``text-describability'' of object detection datasets using the zero-shot image classification accuracy with CLIP. This allows us to categorize various OD datasets with different text-describability and emprically evaluate the FSOD performance of OVD and COD methods within each category. Our findings reveal that: i) there is little difference between OVD and COD for object classes with low text-describability under equal conditions in OD pretraining; and ii) although OVD can learn from more diverse data than OD-specific data, thereby increasing the volume of training data, it can be counterproductive for classes with low-text-describability. These findings provide practitioners with valuable guidance amidst the recent advancements of OVD methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。