医学视觉语言模型可能泄露脱敏影像与报告的关联，需通过微调降低风险。
Cross-modal linkage risk in clinical vision-language models
- 在12万患者数据上测试模型，发现越专业的模型越易通过相似度复原配对报告。
- 强模型在1万个候选中正确召回率达50倍随机水平，且对抗病灶干扰仍有效。
- 仅微调对齐层并加差分隐私，可降61.8%重连率，图像表征能力几乎不变。
在胸片与放射科报告配对数据上训练的视觉语言模型（VLMs）会学习共享嵌入空间，保留图像与报告的实例级对应关系。这在影像与报告分离存储的场景下带来隐私风险：脱敏影像可通过余弦相似度被重新链接至原始报告。本文将此定义为图像到报告的检索任务，并利用已知真实配对的公开数据集作为基准评估风险。在涵盖126,804名患者、共406,241对样本的MIMIC-CXR（43,793个保留对）和外部CheXpert Plus（29,296对）数据集上，发现模型临床专业化程度越高，重连能力越强：最强模型在候选池为100时，正确报告召回率达随机水平的15倍；在10,000个候选中达50倍；在全库规模下仍显著高于随机。该信号在病理匹配的难负样本下依然存在，表明其超越一般诊断类别。为降低风险而不重训练，我们冻结双编码器，仅对定义对齐层的投影头应用差分隐私优化（ε=0.34，δ=6×10⁻⁶）。该方法在MIMIC-CXR上使召回率@1在N=10,000时下降61.8%，并可无须再训练迁移至CheXpert Plus，同时图像侧性能基本保持：14类线性探测分类的宏平均AUROC仅从79.63%降至79.43%。针对共享对齐层的定向差分隐私微调，能显著降低跨模态重连风险，且不明显损害模型的临床实用性。
原文摘要 · Abstract (English)
Vision-language models (VLMs) trained on paired chest radiographs and radiology reports learn a shared embedding space that can preserve instance-level image-report correspondence. This poses a privacy risk in settings where radiographs and reports are deliberately kept separate after acquisition, such as image-only data sharing or access-controlled reports, because a de-identified image may be re-linked to its original narrative report through cosine similarity alone. We formalized this as image-to-report retrieval and used public paired cohorts, in which the true pairing is known by design, as ground-truth benchmarks to audit the risk rather than as the privacy scenario. Evaluating VLMs of increasing clinical specialization on 406,241 paired examples from 126,804 patients across MIMIC-CXR (43,793 held-out pairs) and external CheXpert Plus (29,296 pairs), we found that re-linkage rose systematically with specialization: the strongest VLM retrieved the correct report at 15 times chance at a candidate pool of N = 100, 50 times chance at N = 10,000, and well above chance at full-database scale. The signal persisted under pathology-matched hard negatives that removed disease-label shortcuts, indicating correspondence beyond broad diagnostic categories. To reduce it without retraining, we froze both encoders and applied differentially private optimization only to the projection heads defining the alignment layer (epsilon = 0.34, delta = 6x10-6). This reduced Recall@1 by 61.8% at N = 10,000 on MIMIC-CXR and transferred to CheXpert Plus without retraining, while image-side utility was largely preserved: macro AUROC for linear-probe classification across 14 labels shifted only from 79.63% to 79.43%. Targeted DP finetuning of the shared alignment layer can substantially reduce cross-modal re-linkage without materially degrading the image representations that make these models clinically useful.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。