解决噪声标签下多标签跨模态检索的鲁棒性问题
Semantic-Consistent Bidirectional Contrastive Hashing for Noisy Multi-Label Cross-Modal Retrieval
- 利用跨模态语义一致性判断样本可信度,减轻噪声标签影响
- 基于多标签语义重叠动态生成软对比对,提升相似与相异样本学习效果
- 在四个数据集上均超越现有方法,适合真实噪声场景应用
跨模态哈希(CMH)通过将数据编码为紧凑二进制表示,实现图像与文本等不同模态间的高效检索。尽管近期方法性能优异,但普遍依赖全标注数据集,而真实场景中多标签数据常含噪声标签,严重降低检索效果。此外,现有方法忽视多标签数据中固有的部分语义重叠,限制了模型鲁棒性与泛化能力。为此,我们提出一种新框架——语义一致双向对比哈希(SCBCH),包含两个互补模块:(1) 跨模态语义一致性分类(CSCC),利用跨模态语义一致性估计样本可靠性,降低噪声标签影响;(2) 双向软对比哈希(BSCH),基于多标签语义重叠动态生成软对比样本对,实现跨模态间语义相似与相异样本的自适应对比学习。在四个常用跨模态检索基准上的大量实验验证了该方法的有效性与鲁棒性,在噪声多标签条件下持续优于当前最优方法。
原文摘要 · Abstract (English)
Cross-modal hashing (CMH) facilitates efficient retrieval across different modalities (e.g., image and text) by encoding data into compact binary representations. While recent methods have achieved remarkable performance, they often rely heavily on fully annotated datasets, which are costly and labor-intensive to obtain. In real-world scenarios, particularly in multi-label datasets, label noise is prevalent and severely degrades retrieval performance. Moreover, existing CMH approaches typically overlook the partial semantic overlaps inherent in multi-label data, limiting their robustness and generalization. To tackle these challenges, we propose a novel framework named Semantic-Consistent Bidirectional Contrastive Hashing (SCBCH). The framework comprises two complementary modules: (1) Cross-modal Semantic-Consistent Classification (CSCC), which leverages cross-modal semantic consistency to estimate sample reliability and reduce the impact of noisy labels; (2) Bidirectional Soft Contrastive Hashing (BSCH), which dynamically generates soft contrastive sample pairs based on multi-label semantic overlap, enabling adaptive contrastive learning between semantically similar and dissimilar samples across modalities. Extensive experiments on four widely-used cross-modal retrieval benchmarks validate the effectiveness and robustness of our method, consistently outperforming state-of-the-art approaches under noisy multi-label conditions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。