通过动态融合多模型优势,提升图像检索的精准度与稳定性。
DAFM: Dynamic Adaptive Fusion for Multi-Model Collaboration in Composed Image Retrieval
- 设计动态自适应融合机制,根据任务需求自动分配不同模型权重。
- 在CIRR数据集上达Recall@10 93.21,Rmean 84.43,超越基线最高4.5%。
- 适用于需要跨模态精准检索的场景,如电商商品查找与内容推荐。
组合图像检索(CIR)是一项跨模态任务,旨在通过参考图像和修改文本从大规模数据库中检索目标图像。现有方法多依赖单一模型进行特征融合与相似性匹配,面临两大挑战:一是单模型难以同时兼顾全局与细节,相同权重下易忽略图像与文本间细微关联;二是缺乏动态权重分配,无法自适应发挥各模型互补优势,导致嵌入表示偏离目标,误导最近邻搜索。为此,本文提出动态自适应融合(DAFM)框架,通过协同异构模型并动态重平衡其贡献,不仅提升检索准确率,且性能不受融合顺序影响,体现方法鲁棒性。在CIRR与FashionIQ基准测试中,本方法分别取得Recall@10 93.21、Rmean 84.43,以及FashionIQ平均Rmean 67.48,相较近期强基线提升最高达4.5%,验证了动态多模型协作在CIR中的有效性和通用性。
原文摘要 · Abstract (English)
Composed Image Retrieval (CIR) is a cross-modal task that aims to retrieve target images from large-scale databases using a reference image and a modification text. Most existing methods rely on a single model to perform feature fusion and similarity matching. However, this paradigm faces two major challenges. First, one model alone can't see the whole picture and the tiny details at the same time; it has to handle different tasks with the same weights, so it often misses the small but important links between image and text. Second, the absence of dynamic weight allocation prevents adaptive leveraging of complementary model strengths, so the resulting embedding drifts away from the target and misleads the nearest-neighbor search in CIR. To address these limitations, we propose Dynamic Adaptive Fusion (DAFM) for multi-model collaboration in CIR. Rather than optimizing a single method in isolation, DAFM exploits the complementary strengths of heterogeneous models and adaptively rebalances their contributions. This not only maximizes retrieval accuracy but also ensures that the performance gains are independent of the fusion order, highlighting the robustness of our approach. Experiments on the CIRR and FashionIQ benchmarks demonstrate consistent improvements. Our method achieves a Recall@10 of 93.21 and an Rmean of 84.43 on CIRR, and an average Rmean of 67.48 on FashionIQ, surpassing recent strong baselines by up to 4.5%. These results confirm that dynamic multi-model collaboration provides an effective and general solution for CIR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。