提出双向交叉注意力模型，提升图文检索的语义对齐与效率。
UniCA: Bi-directional Cross-Attention with Positive Similarity Loss for Robust Multi-Modal Retrieval
- 采用双向交叉注意力，主动融合视觉与文本特征。
- 在WebQA上实现最高4.09%的Recall@5提升。
- 轻量数据集降低计算成本，适合实际部署。
多模态检索在处理海量图文数据中日益重要，但现有方法依赖文本编码器自注意力进行隐式融合，难以显式对齐跨模态语义。本文提出UniCA（统一交叉注意力编码器），包含四项创新：1）双向交叉注意力（Bi-CA）模块，在拼接前主动交换视觉与文本标记间的语义信息，更高效捕捉跨模态关联；2）正相似性损失，优化查询与正样本嵌入间的绝对语义接近度；3）轻量化数据集UMR-S10（通用多模态检索样本10%），在保持语义多样性与任务代表性的同时降低计算开销；4）在WebQA基准上验证，UniCA在混合任务与图文任务中均优于基线模型，混合任务下Recall@5提升4.09%，Recall@10提升3.28%，MRR@1提升3.96%。UniCA通过轻量数据集与增强融合机制，提供了高效可靠的多模态检索解决方案。
原文摘要 · Abstract (English)
Multi-modal retrieval has become increasingly critical for handling the growing volume of integrated visual-textual data in real-world applications, but existing frameworks rely on implicit fusion via text encoder self-attention, limiting explicit cross-modal semantic alignment. To address this gap, this paper proposes UniCA (Unified Cross-Attention Encoder), a multi-modal retrieval model with four key innovations: 1) a bi-directional cross-attention (Bi-CA) block that enables active semantic exchange between visual and textual tokens prior to concatenation, capturing inter-modal correlations more efficiently. 2) a Positive Similarity Loss that optimizes absolute semantic proximity between query and positive candidate embeddings. 3) a streamlined dataset UMR-S10 (Universal Multimodal Retrieval Sample 10%) to reduce computational costs while retaining semantic diversity and task representativeness. 4) an experimental validation on the WebQA benchmark demonstrates that UniCA outperforms the baseline model across Hybrid and Image-Text tasks, achieving improvements of up to 4.09% in Recall@5, 3.28% in Recall@10, and 3.96% in MRR@1 for the hybrid task. UniCA provides an efficient and robust solution for multi-modal retrieval, lowering deployment barriers through its lightweight dataset and enhanced fusion mechanism.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。