arXiv:2607.21546cs.CV2026-07

无配对医学影像跨模态知识迁移,提升分割精度

UnDA: Unpaired Domain Alignment for Cross-Modal Knowledge Transfer in Medical Imaging

论文配图:UnDA: Unpaired Domain Alignment for Cross-Modal Knowledge Transfer in Medical Imaging
图 1 · 摘自论文原文
  • 用注意力池化提取语义类令牌,实现跨模态对齐
  • 不确定性加权最优传输抑制噪声,提升迁移鲁棒性
  • 适合无配对标注的医学影像跨模态学习场景

多模态方法在下游任务中通常优于单模态方法,因不同模态提供互补信息,但现实中获取配对临床数据仍具挑战。尽管跨模态知识蒸馏可缓解此问题,现有方法常受限于大的模态差异及源域不确定预测带来的噪声传播。为此,我们提出UnDA,一种锚点引导的无配对跨模态蒸馏框架。该方法引入与主干网络无关的对齐模块,通过基于注意力的池化机制提取语义结构化的类令牌。为确保稳健的知识迁移,提出不确定性加权最优传输(UCT-OT),根据预测置信度动态加权特征级对齐,有效抑制噪声监督。此外,基于每类的ProtoNCE目标保持稳定的原型记忆,强化无配对批次间的全局判别能力。在严格无配对设置下的代表性分割任务评估显示,目标模态的准确率和边界精度均有持续提升,证明了无需配对数据即可在异构数据源间传递有意义的结构化知识。

原文摘要 · Abstract (English)

Multimodal based approaches often outperform single modality approaches in downstream tasks as the different modalities provide complementary information, yet acquiring paired clinical data remains a significant challenge in real world scenarios. While cross-modal knowledge distillation addresses this, existing methods often struggle with large modality gaps and the propagation of noise from uncertain source-domain predictions. To overcome these challenges, we propose UnDA, an anchor-guided framework for unpaired cross-modal distillation. Our approach introduces a backbone-agnostic Alignment Module that extracts semantically structured class tokens via an attention based pooling mechanism. To ensure robust knowledge transfer, we propose Uncertainty-Weighted Optimal Transport (UCT-OT), which dynamically weights feature-level alignment based on prediction confidence, effectively suppressing noisy supervision. Furthermore, a per-class ProtoNCE objective maintains stable prototype memories to enforce global discriminability across unpaired batches. Evaluations on representative segmentation tasks under strictly unpaired settings show consistent improvements in accuracy and boundary precision in the target modality, demonstrating that meaningful structural knowledge can be transferred across heterogeneous data sources without paired datasets.

跨模态医学影像知识蒸馏无配对

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。