首个实时医学影像开放词汇检测模型，能识别未知病灶。
MedROV: Towards Real-Time Open-Vocabulary Detection Across Diverse Medical Imaging Modalities
- 构建多模态大样本数据集Omnis，用伪标签补全标注。
- 相比现有最佳模型提升40 mAP50，实时运行达70 FPS。
- 适合需泛化检测新病灶的临床研发与算法工程师。
医学影像传统目标检测模型受限于封闭集设定，难以识别新类别。开放词汇检测虽可突破此限制，但在医学领域因数据稀缺与图文对齐弱而研究不足。为此，我们提出首个医学影像实时开放词汇检测模型MedROV。为支持开放词汇学习，我们构建了包含60万样本、覆盖九种成像模态的大规模数据集Omnis，并设计伪标签策略处理多源数据中的缺失标注。通过引入大规模预训练基础模型知识，结合对比学习与跨模态表示，有效提升模型对已知及未知结构的检测能力。实验表明，MedROV在平均绝对性能上较此前最优基础模型提升40 mAP50，优于封闭集检测器超过3 mAP50，同时保持70 FPS的实时推理速度，树立了医学图像检测新基准。代码、数据集与训练模型已在https://github.com/toobatehreem/MedROV开源。
原文摘要 · Abstract (English)
Traditional object detection models in medical imaging operate within a closed-set paradigm, limiting their ability to detect objects of novel labels. Open-vocabulary object detection (OVOD) addresses this limitation but remains underexplored in medical imaging due to dataset scarcity and weak text-image alignment. To bridge this gap, we introduce MedROV, the first Real-time Open Vocabulary detection model for medical imaging. To enable open-vocabulary learning, we curate a large-scale dataset, Omnis, with 600K detection samples across nine imaging modalities and introduce a pseudo-labeling strategy to handle missing annotations from multi-source datasets. Additionally, we enhance generalization by incorporating knowledge from a large pre-trained foundation model. By leveraging contrastive learning and cross-modal representations, MedROV effectively detects both known and novel structures. Experimental results demonstrate that MedROV outperforms the previous state-of-the-art foundation model for medical image detection with an average absolute improvement of 40 mAP50, and surpasses closed-set detectors by more than 3 mAP50, while running at 70 FPS, setting a new benchmark in medical detection. Our source code, dataset, and trained model are available at https://github.com/toobatehreem/MedROV.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。