用X光片实现胸部CT多病种检测,提升筛查效率
X2CT-CLIP: Enable Multi-Abnormality Detection in Computed Tomography from Chest Radiography via Tri-Modal Contrastive Learning
- 通过三模态对比学习,将CT图像与报告知识迁移到X光片模型
- 在三个多标签数据集上超越现有方法,少样本适应能力更强
- 适合资源有限地区,让普通X光片具备CT级疾病识别能力
计算机断层扫描(CT)是诊断关键影像模态,但高辐射和长耗时限制其大规模筛查应用。胸片(CXR)更安全便捷,现有基础模型主要聚焦于可见病灶,难以识别深层异常。尽管有研究尝试在模拟胸片上训练分类模型,仍局限于单一病种识别。虽然CT基础模型显著提升了病灶检测能力,但其标签在胸片上的泛化应用仍不理想。本研究提出X2CT-CLIP,一种三模态知识迁移学习框架,通过精心设计的隐空间对齐机制,将3D CT体数据及相应放射科报告的知识,从CT迁移至胸片编码器,实现跨模态联合建模。在三个多标签CT数据集上的评估表明,该方法在跨模态检索、少样本适应和外部验证中均优于现有先进方法。结果表明,融合CT知识的胸片可成为资源受限环境下的高效疾病检测替代方案。
原文摘要 · Abstract (English)
Computed tomography (CT) is a key imaging modality for diagnosis, yet its clinical utility is marred by high radiation exposure and long turnaround times, restricting its use for larger-scale screening. Although chest radiography (CXR) is more accessible and safer, existing CXR foundation models focus primarily on detecting diseases that are readily visible on the CXR. Recently, works have explored training disease classification models on simulated CXRs, but they remain limited to recognizing a single disease type from CT. CT foundation models have also emerged with significantly improved detection of pathologies in CT. However, the generalized application of CT-derived labels on CXR has remained illusive. In this study, we propose X2CT-CLIP, a tri-modal knowledge transfer learning framework that bridges the modality gap between CT and CXR while reducing the computational burden of model training. Our approach is the first work to enable multi-abnormality classification in CT, using CXR, by transferring knowledge from 3D CT volumes and associated radiology reports to a CXR encoder via a carefully designed tri-modal alignment mechanism in latent space. Extensive evaluations on three multi-label CT datasets demonstrate that our method outperforms state-of-the-art baselines in cross-modal retrieval, few-shot adaptation, and external validation. These results highlight the potential of CXR, enriched with knowledge derived from CT, as a viable efficient alternative for disease detection in resource-limited settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。