arXiv:2410.15346cs.CVcs.AI2024-10ICLR

YOLO-RD通过检索外部知识库提升目标检测精度,仅增1%参数就提3%以上性能。

YOLO-RD: Introducing Relevant and Compact Explicit Knowledge to YOLO by Retriever-Dictionary

  • 引入检索词典模块,从视觉与语言模型中提取全局数据知识
  • 在检测任务上提升超过3%的mAP,参数增加不足1%
  • 适用于单阶段与两阶段模型,适合追求高精度轻量部署的开发者

图像中目标的识别与定位是基础挑战,现有模型常过度关注当前输入而忽略数据集整体信息。本文提出一种新型检索-词典(Retriever-Dictionary, RD)模块,使基于YOLO的模型能高效获取由视觉模型(VM)、大语言模型(LLM)或视觉语言模型(VLM)构建的词典中的数据洞察。该模块支持多任务增强,涵盖像素级到图像级的分割、检测与分类。实验表明,引入RD可显著提升模型性能,在对象检测中实现超过3%的平均精度均值(mAP)提升,同时模型参数增加低于1%。该模块不仅优化单阶段检测器,也有效提升两阶段模型及DETR类架构(如Faster R-CNN、Deformable DETR)的性能。代码已开源于https://github.com/henrytsui000/YOLO。

原文摘要 · Abstract (English)

Identifying and localizing objects within images is a fundamental challenge, and numerous efforts have been made to enhance model accuracy by experimenting with diverse architectures and refining training strategies. Nevertheless, a prevalent limitation in existing models is overemphasizing the current input while ignoring the information from the entire dataset. We introduce an innovative Retriever-Dictionary (RD) module to address this issue. This architecture enables YOLO-based models to efficiently retrieve features from a Dictionary that contains the insight of the dataset, which is built by the knowledge from Visual Models (VM), Large Language Models (LLM), or Visual Language Models (VLM). The flexible RD enables the model to incorporate such explicit knowledge that enhances the ability to benefit multiple tasks, specifically, segmentation, detection, and classification, from pixel to image level. The experiments show that using the RD significantly improves model performance, achieving more than a 3\% increase in mean Average Precision for object detection with less than a 1% increase in model parameters. Beyond 1-stage object detection models, the RD module improves the effectiveness of 2-stage models and DETR-based architectures, such as Faster R-CNN and Deformable DETR. Code is released at https://github.com/henrytsui000/YOLO.

目标检测知识增强YOLO轻量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。