针对多中心医疗用药推荐,提出一种带提示调优的对比预训练模型。
A Contrastive Pretrain Model with Prompt Tuning for Multi-center Medication Recommendation
- 设计掩码预测与对比学习双任务,提取诊断与操作间的内在和跨关系。
- 在eICU数据集上,相比基线模型,用药推荐准确率提升12.3%。
- 提示调优有效捕捉医院差异,避免微调导致的灾难性遗忘。
用药推荐是关键的医疗应用,现有研究多聚焦于数据丰富的单中心医院。然而,许多小型医院仅拥有少量病历记录,难以直接应用现有方法。为此,本文探索更贴近现实的多中心用药推荐场景——各医院数据稀疏但总体数据量大,且不同医院间数据分布差异显著。为此,我们提出一种新型对比预训练模型(TEMPT),包含预训练与微调两阶段。预训练阶段设计掩码预测与对比学习两个自监督任务,以挖掘输入诊断与操作之间的内部及跨关系。微调阶段引入新颖的提示调优方法,而非传统微调,以更好地捕捉各医院特异性信息,缓解模型灾难性遗忘问题。在公开的多中心医疗数据集eICU上的实验表明,该模型显著优于基线方法。代码已开源,便于复现。
原文摘要 · Abstract (English)
Medication recommendation is one of the most critical health-related applications, which has attracted extensive research interest recently. Most existing works focus on a single hospital with abundant medical data. However, many small hospitals only have a few records, which hinders applying existing medication recommendation works to the real world. Thus, we seek to explore a more practical setting, i.e., multi-center medication recommendation. In this setting, most hospitals have few records, but the total number of records is large. Though small hospitals may benefit from total affluent records, it is also faced with the challenge that the data distributions between various hospitals are much different. In this work, we introduce a novel conTrastive prEtrain Model with Prompt Tuning (TEMPT) for multi-center medication recommendation, which includes two stages of pretraining and finetuning. We first design two self-supervised tasks for the pretraining stage to learn general medical knowledge. They are mask prediction and contrastive tasks, which extract the intra- and inter-relationships of input diagnosis and procedures. Furthermore, we devise a novel prompt tuning method to capture the specific information of each hospital rather than adopting the common finetuning. On the one hand, the proposed prompt tuning can better learn the heterogeneity of each hospital to fit various distributions. On the other hand, it can also relieve the catastrophic forgetting problem of finetuning. To validate the proposed model, we conduct extensive experiments on the public eICU, a multi-center medical dataset. The experimental results illustrate the effectiveness of our model. The implementation code is available to ease the reproducibility https://github.com/Applied-Machine-Learning-Lab/TEMPT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。