让大模型做目标检测，通过三步推理提升精度
CoT4Det: A Chain-of-Thought Framework for Perception-Oriented Vision-Language Tasks
- 将检测任务拆解为分类、计数、定位三步，更契合大模型推理能力
- 在COCO数据集上mAP从19%提升至33%，小物体召回显著改善
- 无需训练，适配通用视觉语言模型，适合希望提升感知能力的开发者
大型视觉语言模型（LVLMs）在通用视觉问答和光学字符识别等任务中表现优异，但在以感知为核心的任务如目标检测、语义分割和深度估计方面，性能远低于专用模型。例如，Qwen2.5-VL-7B-Instruct在COCO2017验证集上仅达到19% mAP，尤其在密集场景和小物体检测上表现不佳。本文提出检测领域的思维链框架CoT4Det，将感知任务重构为分类、计数、定位三个可解释步骤，更符合LVLM的推理逻辑。大量实验表明，该方法在不损害通用视觉语言能力的前提下显著提升感知性能：使用标准Qwen2.5-VL-7B-Instruct模型，COCO2017 val的mAP由19.0%提升至33.0%，在RefCOCO系列上领先基线+2%，在Flickr30k entities上提升19%。
原文摘要 · Abstract (English)
Large Vision-Language Models (LVLMs) have demonstrated remarkable success in a broad range of vision-language tasks, such as general visual question answering and optical character recognition (OCR). However, their performance on perception-centric tasks -- such as object detection, semantic segmentation, and depth estimation -- remains significantly inferior to that of task-specific expert models. For example, Qwen2.5-VL-7B-Instruct achieves only 19% mAP on COCO2017 val, particularly struggling with dense scenes and small object recall. In this work, we introduce Chain-of-Thought for Detection (CoT4Det), a simple but efficient strategy that reformulates perception tasks into three interpretable steps: classification, counting, and grounding -- each more naturally aligned with the reasoning capabilities of LVLMs. Extensive experiments demonstrate that our method significantly improves perception performance without compromising general vision language capabilities. With a standard Qwen2.5-VL-7B-Instruct, CoT4Det boosts mAP from 19.0% to 33.0% on COCO2017 val and achieves competitive results across a variety of perception benchmarks, outperforming baselines by +2% on RefCOCO series and 19% on Flickr30k entities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。