对比四种隐私保护方法，发现知识蒸馏在临床编码中效果最佳。
How to Train Private Clinical Language Models: A Comparative Study of Privacy-Preserving Pipelines for ICD-9 Coding
- 用知识蒸馏从私密训练的教师模型迁移知识
- 在ε=4~6时恢复63%非私密模型性能
- 适合需要高隐私保障的医疗AI部署
在临床文本上训练的大语言模型可能泄露敏感患者信息，而差分隐私（DP）方法常严重降低诊断准确性。尽管DP优化和文本生成进展迅速，但尚不清楚哪种隐私保护策略最适合临床语言任务。本文首次系统性地对比了四种从医院出院记录自动诊断编码的训练流程。所有流程使用相同的10亿参数模型和相同的隐私预算来预测ICD-9编码。在中等和宽松隐私预算（ε ∈ {4, 6}）下，从私密训练的教师模型进行知识蒸馏的表现优于直接使用DP-SGD和DP合成数据训练，恢复了高达63%的非私密模型性能，同时保持强实证隐私性（成员推断攻击AUC ≈ 0.5）。结果揭示了不同架构间隐私-效用权衡的巨大差异，并指出知识蒸馏是实现隐私保护临床自然语言处理最实用的路径。
原文摘要 · Abstract (English)
Large language models trained on clinical text risk exposing sensitive patient information, yet differential privacy (DP) methods often severely degrade the diagnostic accuracy needed for deployment. Despite rapid progress in DP optimisation and text generation, it remains unclear which privacy-preserving strategy actually works best for clinical language tasks. We present the first systematic head-to-head comparison of four training pipelines for automated diagnostic coding from hospital discharge summaries. All pipelines use identical 1B-parameter models and matched privacy budgets to predict ICD-9 codes. At moderate and relaxed privacy budgets ($\varepsilon \in \{4, 6\}$), knowledge distillation from DP-trained teachers outperforms both direct DP-SGD and DP-synthetic data training, recovering up to 63\% of the non-private performance whilst maintaining strong empirical privacy (membership-inference AUC $\approx$ 0.5). These findings expose large differences in the privacy-utility trade-off across architectures and identify knowledge distillation as the most practical route to privacy-preserving clinical NLP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。