用最优传输对齐多视角医学图像特征,提升分类鲁棒性
OTCHA: Optimal Transport-driven Confidence-aware Latent Hub Alignment for Multi-View Medical Image Classification

- 引入可学习的共享隐空间枢纽令牌,通过最优传输匹配各视角图像块
- 在三个数据集上优于基线方法,尤其在不同解剖结构和视角下表现稳定
- 适合处理带噪声或无关背景的医学图像多视角分类任务
多视角成像(如乳腺钼靶和胸部X光)是临床常规。但医学图像常未配准,包含视角特异性伪影或无关背景,掩盖诊断关键信息。现有方法直接融合各视角表征,导致无关内容污染融合嵌入,降低在不同视角配置下的鲁棒性。本文提出OTCHA,一种基于最优传输(OT)的置信度感知隐空间枢纽令牌对齐模块,用于多视角分类前的特征精炼。OTCHA引入一组跨视角共享的可学习隐空间枢纽令牌。对每个视角,计算图像块令牌与枢纽令牌间的最优传输计划,联合考虑特征相似性和几何结构,并引入条件尘箱(token-conditional dustbins)实现部分匹配并剔除无关令牌。生成的传输计划提供逐令牌匹配置信度,用于门控枢纽引导的消息传递,并加权一个新型最优传输表征对齐损失以稳定精炼过程。在三个多视角医学图像数据集上的实验表明,相比竞争基线,本方法在多样解剖结构和视角配置下均取得一致提升。代码已开源:https://github.com/labhai/OTCHA。
原文摘要 · Abstract (English)
Multi-view imaging, such as mammography and chest radiography, is a standard component of clinical practice. However, medical images are often unregistered and contain view-specific artifacts or irrelevant background cues that can obscure diagnostically relevant findings. Many existing methods directly fuse per-view representations, allowing such irrelevant content to contaminate the fused embedding and reducing robustness under varying view configurations. We propose OTCHA, a confidence-aware latent hub token alignment module based on optimal transport (OT) that refines patch tokens before fusion for multi-view classification. OTCHA introduces a set of learnable latent hub tokens shared across views. For each view, we compute an OT plan between patch tokens and hub tokens that jointly considers feature similarity and geometry, and augment the OT formulation with token-conditional dustbins to enable partial matching and discard irrelevant tokens. The resulting transport plan provides token-wise matching confidence, which gates hub-mediated message passing and weights a novel optimal-transport-based representation alignment loss to stabilize refinement. Experiments on three multi-view medical image datasets demonstrate consistent improvements over competing baselines across diverse anatomies and view configurations. Our code is available at https://github.com/labhai/OTCHA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。