一个模型搞定多种医学影像重识别，提升诊断与隐私保护效率。
Towards All-in-One Medical Image Re-Identification
- 用连续模态适配器动态调整模型参数，统一处理不同医学影像。
- 在11个数据集上优于25个基础模型和8个多模态大模型。
- 适合医疗个性化诊断与隐私保护场景，代码开源可复现。
医学影像重识别（MedReID）在个性化医疗和隐私保护中具有关键作用，但研究仍不充分。本文提出一个全面基准与统一模型。为应对多种医学模态，设计了连续模态参数适配器（ComPA），将医学内容编码为连续模态表示，并在运行时动态调整通用模型的参数，实现单一模型对多模态数据的自适应学习。同时，通过与预训练医学基础模型在差异特征层面的对齐，融入医学先验知识。相比单图特征，建模图像间差异更契合重识别任务需求。在11个图像数据集上评估，超越25个基础模型与8个大型多模态语言模型，表现一致领先。进一步应用于历史增强个性化诊断与医疗隐私保护，验证实际价值。代码与模型已开源。
原文摘要 · Abstract (English)
Medical image re-identification (MedReID) is under-explored so far, despite its critical applications in personalized healthcare and privacy protection. In this paper, we introduce a thorough benchmark and a unified model for this problem. First, to handle various medical modalities, we propose a novel Continuous Modality-based Parameter Adapter (ComPA). ComPA condenses medical content into a continuous modality representation and dynamically adjusts the modality-agnostic model with modality-specific parameters at runtime. This allows a single model to adaptively learn and process diverse modality data. Furthermore, we integrate medical priors into our model by aligning it with a bag of pre-trained medical foundation models, in terms of the differential features. Compared to single-image feature, modeling the inter-image difference better fits the re-identification problem, which involves discriminating multiple images. We evaluate the proposed model against 25 foundation models and 8 large multi-modal language models across 11 image datasets, demonstrating consistently superior performance. Additionally, we deploy the proposed MedReID technique to two real-world applications, i.e., history-augmented personalized diagnosis and medical privacy protection. Codes and model is available at \href{https://github.com/tianyuan168326/All-in-One-MedReID-Pytorch}{https://github.com/tianyuan168326/All-in-One-MedReID-Pytorch}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。