高效融合病理图像与病历文本,提升胎盘疾病诊断准确率
Efficient Multi-Slide Visual-Language Feature Fusion for Placental Disease Classification
- 两阶段补丁筛选结合无参数与可学习压缩,兼顾效率与特征保留
- 混合多模态融合利用自适应图学习增强病理特征,文本报告补充全局上下文
- 在自建及公开数据集上均达最优性能,适合医学影像诊断研究者参考
通过全切片图像(WSIs)准确预测胎盘疾病对预防严重母体和胎儿并发症至关重要。然而,由于数据量巨大,WSI分析面临显著计算挑战。现有分类方法存在两大关键局限:(1) 补丁选择策略不足,或损害性能,或无法充分降低计算负担;(2) 基于补丁级处理导致全局组织学上下文丢失。为此,我们提出一种面向患者级胎盘疾病诊断的高效多模态框架EmmPD。该方法引入两阶段补丁选择模块,结合无参数与可学习压缩策略,优化计算效率与关键特征保留的平衡。同时,设计混合多模态融合模块,利用自适应图学习增强病理特征表示,并融合文本医学报告以丰富全局上下文理解。在自建患者级胎盘数据集及两个公开数据集上的大量实验表明,本方法实现最先进的诊断性能。代码已开源:https://github.com/ECNU-MultiDimLab/EmmPD。
原文摘要 · Abstract (English)
Accurate prediction of placental diseases via whole slide images (WSIs) is critical for preventing severe maternal and fetal complications. However, WSI analysis presents significant computational challenges due to the massive data volume. Existing WSI classification methods encounter critical limitations: (1) inadequate patch selection strategies that either compromise performance or fail to sufficiently reduce computational demands, and (2) the loss of global histological context resulting from patch-level processing approaches. To address these challenges, we propose an Efficient multimodal framework for Patient-level placental disease Diagnosis, named EmmPD. Our approach introduces a two-stage patch selection module that combines parameter-free and learnable compression strategies, optimally balancing computational efficiency with critical feature preservation. Additionally, we develop a hybrid multimodal fusion module that leverages adaptive graph learning to enhance pathological feature representation and incorporates textual medical reports to enrich global contextual understanding. Extensive experiments conducted on both a self-constructed patient-level Placental dataset and two public datasets demonstrating that our method achieves state-of-the-art diagnostic performance. The code is available at https://github.com/ECNU-MultiDimLab/EmmPD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。