arXiv:2602.16245cs.CV2026-02中稿 · the IEEE/CVF Winte…被引 1

提出轻量级多模态融合网络,提升医学影像分析精度与效率

HyPCA-Net: Advancing Multimodal Fusion in Medical Image Analysis

  • 采用并行-级联双注意力结构,兼顾模态特异性与共享表征学习
  • 在10个数据集上最高提升5.2%性能,计算成本降低73.1%
  • 适合资源受限场景下的多疾病医学影像分析

多模态融合框架通过整合如MRI、CT等不同医学影像模态,在皮肤癌检测、痴呆诊断和脑肿瘤预测等任务中展现出巨大潜力。然而,现有方法存在两大挑战:一是模型计算开销大,限制了在低资源环境的应用;二是普遍采用级联注意力模块,导致模态间信息传递时易丢失,难以有效捕捉跨模态的鲁棒共享表示,影响多疾病分析任务的泛化能力。为此,本文提出混合并行-级联注意力网络(HyPCA-Net),包含两个核心创新模块:(a) 轻量级残差自适应学习注意力块,用于捕获精细的模态特异性表示;(b) 双视角级联注意力块,旨在学习跨多样模态的稳健共享表示。在十个公开数据集上的大量实验表明,HyPCA-Net显著优于现有先进方法,性能最高提升5.2%,计算成本最多降低73.1%。代码已开源:https://github.com/misti1203/HyPCA-Net。

原文摘要 · Abstract (English)

Multimodal fusion frameworks, which integrate diverse medical imaging modalities (e.g., MRI, CT), have shown great potential in applications such as skin cancer detection, dementia diagnosis, and brain tumor prediction. However, existing multimodal fusion methods face significant challenges. First, they often rely on computationally expensive models, limiting their applicability in low-resource environments. Second, they often employ cascaded attention modules, which potentially increase risk of information loss during inter-module transitions and hinder their capacity to effectively capture robust shared representations across modalities. This restricts their generalization in multi-disease analysis tasks. To address these limitations, we propose a Hybrid Parallel-Fusion Cascaded Attention Network (HyPCA-Net), composed of two core novel blocks: (a) a computationally efficient residual adaptive learning attention block for capturing refined modality-specific representations, and (b) a dual-view cascaded attention block aimed at learning robust shared representations across diverse modalities. Extensive experiments on ten publicly available datasets exhibit that HyPCA-Net significantly outperforms existing leading methods, with improvements of up to 5.2% in performance and reductions of up to 73.1% in computational cost. Code: https://github.com/misti1203/HyPCA-Net.

多模态融合医学影像轻量化注意力机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。