专为医疗时间序列设计的高效变压器,提升重症预测准确率。
METHOD: Modular Efficient Transformer for Health Outcome Discovery
- 用患者感知注意力避免信息泄露,支持批量处理
- 自适应滑动窗口捕捉多尺度时间依赖,预测高危病例更准
- 结构类似U-Net,动态跳跃连接适合长序列,临床部署稳定
近年来,变压器架构在自然语言处理中取得突破,但在医疗领域应用面临独特挑战:患者时间序列具有不规则采样、可变时间依赖和复杂上下文关系。本文提出/method~(Modular Efficient Transformer for Health Outcome Discovery),一种专为电子病历临床序列建模设计的新架构。该方法包含三项创新:(1) 患者感知注意力机制,在防止信息泄露的同时实现高效批处理;(2) 自适应滑动窗口注意力,捕捉多尺度时间依赖;(3) 受U-Net启发的动态跳跃连接结构,提升长序列处理能力。在MIMIC-IV数据库上的评估显示,/method~在预测高危病例方面持续优于当前最优模型/ethos,且在不同推理长度下表现稳定,适用于临床部署。嵌入分析表明,/method~更好地保留了医学概念间的临床层级与关联。结果表明,/method~是面向医疗应用优化的变压器架构的重要进展,兼具更高精度、临床相关性与计算效率。
原文摘要 · Abstract (English)
Recent advances in transformer architectures have revolutionised natural language processing, but their application to healthcare domains presents unique challenges. Patient timelines are characterised by irregular sampling, variable temporal dependencies, and complex contextual relationships that differ substantially from traditional language tasks. This paper introduces \METHOD~(Modular Efficient Transformer for Health Outcome Discovery), a novel transformer architecture specifically designed to address the challenges of clinical sequence modelling in electronic health records. \METHOD~integrates three key innovations: (1) a patient-aware attention mechanism that prevents information leakage whilst enabling efficient batch processing; (2) an adaptive sliding window attention scheme that captures multi-scale temporal dependencies; and (3) a U-Net inspired architecture with dynamic skip connections for effective long sequence processing. Evaluations on the MIMIC-IV database demonstrate that \METHOD~consistently outperforms the state-of-the-art \ETHOS~model, particularly in predicting high-severity cases that require urgent clinical intervention. \METHOD~exhibits stable performance across varying inference lengths, a crucial feature for clinical deployment where patient histories vary significantly in length. Analysis of learned embeddings reveals that \METHOD~better preserves clinical hierarchies and relationships between medical concepts. These results suggest that \METHOD~represents a significant advancement in transformer architectures optimised for healthcare applications, providing more accurate and clinically relevant predictions whilst maintaining computational efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。