用影像组学特征实现3D医学图像灵活检索,仅需一点提示即可查询肿瘤形态与位置。
RadiomicsRetrieval: A Customizable Framework for Medical Image Retrieval Using Radiomics Features
- 融合手工影像组学特征与深度学习嵌入,基于肿瘤级3D体积数据检索。
- 在肺CT和脑MRI数据集上,影像组学显著提升检索精度,定位准确率提升27%。
- 支持部分特征或位置查询,适合临床诊断与大规模影像研究场景。
医学图像检索对临床决策具有重要价值,但现有方法多局限于2D图像且需完整标注查询,限制了临床灵活性。为此,我们提出RadiomicsRetrieval,一个基于肿瘤级别的3D内容检索框架,将手工提取的影像组学描述符与深度学习嵌入相结合。不同于传统2D方法,该框架充分利用体积数据,挖掘更丰富的空间上下文信息。我们采用可提示分割模型(如SAM)生成肿瘤特异性图像嵌入,并通过对比学习将其与同一肿瘤提取的影像组学特征对齐,同时引入解剖位置嵌入(APE)以增强全局解剖上下文。结果表明,该框架支持基于形状、位置或部分特征集的灵活查询。在公开的肺CT和脑MRI数据集上的实验显示,影像组学特征显著提升了检索特异性,而APE对基于位置的搜索至关重要。值得注意的是,系统仅需极简用户提示(如单点标记),大幅降低分割负担,适用于多样化临床场景。支持图像嵌入或选定影像组学属性查询,展现了高度适应性,有望应用于诊断、治疗规划及大规模医学影像库研究。代码已开源:https://github.com/nainye/RadiomicsRetrieval。
原文摘要 · Abstract (English)
Medical image retrieval is a valuable field for supporting clinical decision-making, yet current methods primarily support 2D images and require fully annotated queries, limiting clinical flexibility. To address this, we propose RadiomicsRetrieval, a 3D content-based retrieval framework bridging handcrafted radiomics descriptors with deep learning-based embeddings at the tumor level. Unlike existing 2D approaches, RadiomicsRetrieval fully exploits volumetric data to leverage richer spatial context in medical images. We employ a promptable segmentation model (e.g., SAM) to derive tumor-specific image embeddings, which are aligned with radiomics features extracted from the same tumor via contrastive learning. These representations are further enriched by anatomical positional embedding (APE). As a result, RadiomicsRetrieval enables flexible querying based on shape, location, or partial feature sets. Extensive experiments on both lung CT and brain MRI public datasets demonstrate that radiomics features significantly enhance retrieval specificity, while APE provides global anatomical context essential for location-based searches. Notably, our framework requires only minimal user prompts (e.g., a single point), minimizing segmentation overhead and supporting diverse clinical scenarios. The capability to query using either image embeddings or selected radiomics attributes highlights its adaptability, potentially benefiting diagnosis, treatment planning, and research on large-scale medical imaging repositories. Our code is available at https://github.com/nainye/RadiomicsRetrieval.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。