arXiv:2511.04998cs.LGcs.AI2025-11

用双位置编码增强Transformer,精准预测酒精药物滥用风险。

BiPETE: A Bi-Positional Embedding Transformer Encoder for Risk Assessment of Alcohol and Substance Use Disorder with Electronic Health Records

  • 引入旋转与正弦位置编码,捕捉就诊时间相对关系和顺序。
  • 在抑郁和创伤后应激障碍队列中AUPRC提升34%和50%。
  • 可解释性强,识别出炎症、代谢等关键风险指标。

基于Transformer的深度学习模型在电子健康记录(EHR)疾病风险预测中表现良好,但不规则就诊间隔和缺乏统一结构使其难以建模时序依赖。本文提出双位置编码Transformer编码器(BiPETE),融合旋转位置编码以表示相对就诊时间,以及正弦位置编码以保留就诊顺序。BiPETE在两个精神健康队列(抑郁症和创伤后应激障碍)的EHR数据上进行训练,用于预测酒精与物质使用障碍(ASUD)风险,无需大规模预训练。实验表明,其在抑郁症和PTSD队列中分别将精确率-召回率曲线下面积(AUPRC)提升34%和50%。消融实验证明双位置编码策略有效。通过集成梯度法进行可解释性分析,识别出与ASUD风险及保护相关的临床特征,如异常炎症、血液学和代谢标志物,特定药物及共病情况。这些关键特征有助于深入理解风险评估机制,并为干预提供线索。本研究构建了一个高效且可解释的EHR疾病风险预测框架。

原文摘要 · Abstract (English)

Transformer-based deep learning models have shown promise for disease risk prediction using electronic health records(EHRs), but modeling temporal dependencies remains a key challenge due to irregular visit intervals and lack of uniform structure. We propose a Bi-Positional Embedding Transformer Encoder or BiPETE for single-disease prediction, which integrates rotary positional embeddings to encode relative visit timing and sinusoidal embeddings to preserve visit order. Without relying on large-scale pretraining, BiPETE is trained on EHR data from two mental health cohorts-depressive disorder and post-traumatic stress disorder (PTSD)-to predict the risk of alcohol and substance use disorders (ASUD). BiPETE outperforms baseline models, improving the area under the precision-recall curve (AUPRC) by 34% and 50% in the depression and PTSD cohorts, respectively. An ablation study further confirms the effectiveness of the dual positional encoding strategy. We apply the Integrated Gradients method to interpret model predictions, identifying key clinical features associated with ASUD risk and protection, such as abnormal inflammatory, hematologic, and metabolic markers, as well as specific medications and comorbidities. Overall, these key clinical features identified by the attribution methods contribute to a deeper understanding of the risk assessment process and offer valuable clues for mitigating potential risks. In summary, our study presents a practical and interpretable framework for disease risk prediction using EHR data, which can achieve strong performance.

风险预测Transformer电子病历可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。