用视觉变压器实现胸部X光片多病种分类与弱监督定位,精度超前。
CLARiTy: A Vision Transformer for Multi-Label Classification and Weakly-Supervised Localization of Chest X-ray Pathologies
- 通过类别专用令牌生成注意力图,结合解剖先验进行背景抑制。
- 在NIH ChestX-ray14上对8种病灶实现领先弱监督定位,提升50.7%。
- 轻量版适合资源受限场景,小病灶如结节定位效果尤为突出。
胸部X光片的解读面临多标签病理分类与空间定位的挑战,二者需要不同粒度的标注,但区域级(密集)标注稀缺。本文提出CLARiTy(Class Localizing and Attention Refining Image Transformer),一种基于视觉变压器的联合多标签分类与弱监督定位模型。该模型采用多个类别特定令牌生成判别性注意力图,并引入SegmentCAM模块,利用显式解剖先验实现前景分割与背景抑制。在仅使用图像级标签的NIH ChestX-ray14数据集上训练,通过从ConvNeXtV2教师模型中蒸馏知识以提升效率。在官方NIH划分上,CLARiTy-S-16-512配置在14种病理的分类任务中表现优异,在8种病理的弱监督定位任务中达到当前最优水平,较之前方法提升50.7%。尤其在小病灶(如结节、肿块)上表现显著。低分辨率版本CLARiTy-S-16-224兼具高效率与强性能,适用于低资源环境。消融实验验证了SegmentCAM、DINO预训练、正交类别令牌损失和注意力池化的作用。相比传统CNN-ViT混合架构,CLARiTy利用视觉变压器自注意力捕捉全局上下文,并通过卷积背景抑制获得更精确、噪声更低的热图。
原文摘要 · Abstract (English)
The interpretation of chest X-rays (CXRs) poses significant challenges, particularly in achieving accurate multi-label pathology classification and spatial localization. These tasks demand different levels of annotation granularity but are frequently constrained by the scarcity of region-level (dense) annotations. We introduce CLARiTy (Class Localizing and Attention Refining Image Transformer), a vision transformer-based model for joint multi-label classification and weakly-supervised localization of thoracic pathologies. CLARiTy employs multiple class-specific tokens to generate discriminative attention maps, and a SegmentCAM module for foreground segmentation and background suppression using explicit anatomical priors. Trained on image-level labels from the NIH ChestX-ray14 dataset, it leverages distillation from a ConvNeXtV2 teacher for efficiency. Evaluated on the official NIH split, the CLARiTy-S-16-512 (a configuration of CLARiTy), achieves competitive classification performance across 14 pathologies, and state-of-the-art weakly-supervised localization performance on 8 pathologies, outperforming prior methods by 50.7%. In particular, pronounced gains occur for small pathologies like nodules and masses. The lower-resolution variant of CLARiTy, CLARiTy-S-16-224, offers high efficiency while decisively surpassing baselines, thereby having the potential for use in low-resource settings. An ablation study confirms contributions of SegmentCAM, DINO pretraining, orthogonal class token loss, and attention pooling. CLARiTy advances beyond CNN-ViT hybrids by harnessing ViT self-attention for global context and class-specific localization, refined through convolutional background suppression for precise, noise-reduced heatmaps.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。