融合头颅CT与临床数据,提升脑水肿自动识别准确率。
A Multimodal Deep Learning Framework for Edema Classification Using HCT and Clinical Data
- 用交叉注意力动态融合影像与患者信息,实现可解释的多模态融合。
- 在真实临床数据上达到87.32%准确率,优于单一模态和现有方法。
- 适合医疗AI研究者与放射科医生参考,推动临床辅助诊断落地。
我们提出AttentionMixer,一个统一的深度学习框架,用于结合头颅CT(HCT)与常规临床元数据进行脑水肿多模态检测。虽然HCT提供丰富的空间信息,但年龄、检验值、扫描时间等临床变量可补充关键上下文,常被忽略或简单拼接。AttentionMixer通过自监督视觉变压器自动编码器(ViT-AE++)对HCT体积进行编码,无需大规模标注数据;将临床元数据映射至同一特征空间,作为交叉注意力模块中的键与值,而HCT特征向量作为查询。该机制使网络能根据患者特异性上下文动态调节影像特征,并提供可解释的融合过程。随后,轻量级MLP-Mixer对融合表示进行精炼,以低参数开销建模全局依赖。缺失或不完整的元数据通过可学习嵌入处理,增强对真实临床数据质量的鲁棒性。我们在经过专家标注的脑HCT队列上进行五折交叉验证,结果表明,AttentionMixer在准确率(87.32%)、精确率(92.10%)、F1分数(85.37%)和AUC(94.14%)上均优于强基线模型(仅用HCT、仅用元数据及先前多模态方法)。消融实验验证了交叉注意力与MLP-Mixer的贡献,置换分析揭示了驱动预测的临床相关变量。结果表明,结构化且可解释的多模态融合能显著提升临床实践中的水肿检测效果。
原文摘要 · Abstract (English)
We propose AttentionMixer, a unified deep learning framework for multimodal detection of brain edema that combines structural head CT (HCT) with routine clinical metadata. While HCT provides rich spatial information, clinical variables such as age, laboratory values, and scan timing capture complementary context that might be ignored or naively concatenated. AttentionMixer is designed to fuse these heterogeneous sources in a principled and efficient manner. HCT volumes are first encoded using a self-supervised Vision Transformer Autoencoder (ViT-AE++), without requiring large labeled datasets. Clinical metadata are mapped into the same feature space and used as keys and values in a cross-attention module, where HCT-derived feature vector serves as queries. This cross-attention fusion allows the network to dynamically modulate imaging features based on patient-specific context and provides an interpretable mechanism for multimodal integration. A lightweight MLP-Mixer then refines the fused representation before final classification, enabling global dependency modeling with substantially reduced parameter overhead. Missing or incomplete metadata are handled via a learnable embedding, promoting robustness to real-world clinical data quality. We evaluate AttentionMixer on a curated brain HCT cohort with expert edema annotations using five-fold cross-validation. Compared with strong HCT-only, metadata-only, and prior multimodal baselines, AttentionMixer achieves superior performance (accuracy 87.32%, precision 92.10%, F1-score 85.37%, AUC 94.14%). Ablation studies confirm the benefit of both cross-attention and MLP-Mixer refinement, and permutation-based metadata importance analysis highlights clinically meaningful variables driving predictions. These results demonstrate that structured, interpretable multimodal fusion can substantially improve edema detection in clinical practice.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。