用差分隐私保护医疗影像报告分类,兼顾隐私与准确率。
Learning to Diagnose Privately: DP-Powered LLMs for Radiology Report Classification
- 结合差分隐私与低秩适配(DP-LoRA),安全微调大模型
- 在MIMIC-CXR上达0.89加权F1,接近非私有方法
- 适合医疗数据隐私敏感场景的模型部署
大型语言模型(LLMs)在教育、医疗和金融等领域广泛应用。在医疗中,它们支持疾病诊断、异常分类和临床决策。其中,放射科报告中的多异常分类对临床工作流自动化和生物医学研究至关重要。凭借强大的自然语言处理能力,LLMs可高效处理非结构化医学文本,减轻人工报告分析负担。为提升性能,常使用机构特有敏感数据(如放射科报告)对LLMs进行微调,但带来隐私风险:模型可能记忆训练数据,易受数据提取攻击;共享微调后模型也可能泄露患者信息。尽管医疗文本分类中对LLMs兴趣增长,针对多异常分类的隐私保护微调仍研究不足。为此,本文提出一种面向自由文本放射科报告多异常分类的差分隐私(DP)微调框架。该方法将差分隐私与低秩适配(LoRA)结合,实现对敏感临床数据的安全微调。进一步利用大模型生成标签训练小型模型,确保在强隐私保障下高效推理。在MIMIC-CXR和CT-RATE数据集上的实验表明,所提DP-LoRA框架在不同隐私预算下均有效。在MIMIC-CXR上,适度隐私预算下加权F1最高达0.89,接近非私有LoRA(0.90)和全参数微调(0.96),证明强隐私可实现且性能损失较小。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly adopted across domains such as education, healthcare, and finance. In healthcare, LLMs support tasks including disease diagnosis, abnormality classification, and clinical decision-making. Among these, multi-abnormality classification of radiology reports is critical for clinical workflow automation and biomedical research. Leveraging strong natural language processing capabilities, LLMs enable efficient processing of unstructured medical text and reduce the administrative burden of manual report analysis. To improve performance, LLMs are often fine-tuned on private, institution-specific datasets such as radiology reports. However, this raises significant privacy concerns: LLMs may memorize training data and become vulnerable to data extraction attacks, while sharing fine-tuned models risks exposing sensitive patient information. Despite growing interest in LLMs for medical text classification, privacy-preserving fine-tuning for multi-abnormality classification remains underexplored. To address this gap, we propose a differentially private (DP) fine-tuning framework for multi-abnormality classification from free-text radiology reports. Our approach integrates differential privacy with Low-Rank Adaptation (LoRA) to efficiently fine-tune LLMs on sensitive clinical data while mitigating leakage risks. We further employ labels generated by a larger LLM to train smaller models, enabling efficient inference under strong privacy guarantees. Experiments on MIMIC-CXR and CT-RATE demonstrate the effectiveness of our DP-LoRA framework across varying privacy regimes. On MIMIC-CXR, our method achieves weighted F1-scores up to 0.89 under moderate privacy budgets, approaching non-private LoRA (0.90) and full fine-tuning (0.96), confirming that strong privacy can be achieved with only modest performance trade-offs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。