用通道注意力引导跨模态知识蒸馏，让小模型也能精准理解图文描述。
Channel Attention-Guided Cross-Modal Knowledge Distillation for Referring Image Segmentation
- 通过通道注意力捕捉视觉与语言的细粒度关联，指导知识迁移。
- 在两个公开数据集上，学生模型性能显著提升，推理无额外参数开销。
- 适合资源受限场景，兼顾效果与部署效率，尤其适用于轻量化应用。
指代图像分割（RIS）需根据语言描述准确分割图像中的目标区域，是融合视觉与语言的跨模态任务。现有RIS方法通常依赖大规模视觉-语言编码模型以提升性能，但其庞大的参数量严重限制了在计算资源有限场景下的部署。为此，本文提出一种通道注意力引导的跨模态知识蒸馏方法，将教师网络学习到的视觉与语言间的高阶细粒度关联，以及各通道所表征语义成分间的关联，迁移到学生网络中。相较于传统的像素级关系蒸馏，该方法不仅使学生网络有效学习教师知识，还保留部分自主学习能力，缓解了学习偏差的传递。在两个公开数据集上的实验表明，所提方法在推理阶段不引入额外参数，可显著提升学生模型性能。
原文摘要 · Abstract (English)
Referring image segmentation (RIS) requires accurate segmentation of target regions in images according to language descriptions, which is a cross-modal task integrating vision and language. Existing RIS methods typically employ large-scale vision and language encoding models to improve performance, but their enormous parameter size severely restricts deployment in scenarios with limited computing resources. To solve this problem, this paper proposes a channel attention-guided cross-modal knowledge distillation method, which transfers the high-order fine-grained correlations between vision and language learned by the teacher network, as well as the correlations between semantic components represented by each channel, to the student network. Compared with the traditional pixel-wise relational distillation, this method not only enables the student to learn the knowledge of the teacher, but also retains part of its independent learning ability, alleviating the transfer of learning bias. Experimental results on two public datasets show that the proposed distillation method does not introduce additional parameters during inference and can achieve significant performance improvement for the student model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。