arXiv:2607.09481cs.CVcs.AI2026-07

提出可复用的语言引导模块,让不同图像和文本模型自由搭配。

Decoupling Language Guidance from Backbones for Text-Guided Medical Segmentation

论文配图:Decoupling Language Guidance from Backbones for Text-Guided Medical Segmentation
图 1 · 摘自论文原文
  • 设计分层适配器,通过特征接口注入语义引导。
  • 跨4个数据集提升分割效果,计算开销小。
  • 适合需要灵活更换视觉/语言模型的研究者使用。

文本引导的医学图像分割利用临床语义提升病灶边界识别,但现有模型将跨模态融合、监督方式与解码器设计耦合于特定架构,导致语言引导模块难以在异构视觉与文本主干间复用,且更换编码器时需重设计网络。本文提出BTHA框架,一种可迁移的分层适配器结构,以稳定特征级接口为核心:给定多尺度视觉特征与文本表示,通过保持解码器张量契约的形状保真适配器注入语义引导。为提升接口有效性,引入分阶段粗到细监督策略,分解为全局图文对齐、多尺度辅助定位与边界感知最终掩码优化。进一步设计尺度自适应门控语义引导(SAGSG)适配器,其中分辨率特异性门控控制文本注入,通道重校准抑制冗余跨模态响应。在多种视觉与文本主干上评估显示,同一适配器与监督设计对卷积与基于Transformer的视觉编码器及不同语言编码器均有效。四个公开数据集实验表明,BTHA在小幅计算开销下超越强基线模型。

原文摘要 · Abstract (English)

Text-guided medical image segmentation leverages clinical semantics to improve lesion delineation, yet many existing models bind cross-modal fusion, supervision, and decoder design into a task-specific architecture. Such tight coupling makes it difficult to reuse language guidance modules across heterogeneous vision and text backbones, and often requires redesigning the network when the encoder pair changes. This paper presents BTHA, a backbone-transferable hierarchical adapter framework for text-guided medical image segmentation. BTHA is built around a stable feature-level interface: given multi-scale visual features and a text representation, it injects semantic guidance through shape-preserving adapters while maintaining the decoder-side tensor contract. To make this interface effective, we introduce a Hierarchical Coarse-to-Fine Supervision Strategy that decomposes learning into global image-text alignment, multi-scale auxiliary localization, and boundary-aware final mask refinement. We further design a Scale-Adaptive Gated Semantic Guidance (SAGSG) adapter, where resolution-specific gates adaptively control textual injection and channel recalibration suppresses redundant cross-modal responses. Evaluations across diverse vision and text backbones show that the same adapter and supervision design remains effective across convolutional and transformer-based visual encoders as well as different language encoders. Experiments on four public datasets further demonstrate that BTHA improves strong text-guided baselines with modest computational overhead.

医学分割文本引导可迁移适配器

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。