通过属性邻居关系提升图像检索精度，解决外观相似但属性不同的难题。
COMBINER: Composed Image Retrieval Guided by Attribute-based Neighbor Relations
- 基于属性原型构建跨模态统一表征，解耦语义纠缠
- 引入双关系建模，捕捉样本间属性相似性与邻近关系
- 首个关注外观相似但属性无关样本的CIR方法，适合细粒度检索场景
组成式图像检索（CIR）是一项挑战性任务，旨在通过多模态输入定位特定图像。尽管近年来取得进展，现有方法常忽视外观相似但属性不同的情况，影响多模态特征融合与相似性建模。为此，本文提出一种基于属性原型的统一表征框架。针对三个核心问题——属性级语义纠缠、模态间不一致、监督信号缺失——设计了COMBINER网络：首先引入自适应语义解耦模块，从多模态原始特征中分离属性特征；其次提出基于原型的统一组合模块，构建跨模态统一原型（CUP）以实现特征融合；最后设计双关系建模模块，基于属性相似性挖掘成对关系与邻域关系。相较于传统邻居关系建模方法，COMBINER首次关注外观相似但属性无关的样本，通过属性原型驱动的相似性度量，更精准理解样本间语义关联。在三个基准数据集上的实验证明其有效性。代码已开源：https://github.com/Lee-zixu/COMBINER。
原文摘要 · Abstract (English)
Composed Image Retrieval (CIR) represents a challenging retrieval task that targets locating specific images through multimodal inputs. Despite recent progress in CIR techniques, prior approaches often overlook cases where images appear visually alike yet differ in attributes, potentially undermining both multimodal feature fusion and similarity modeling. To mitigate this limitation, we design a unified representation of cross-modal features based on attribute prototypes. Nevertheless, the task is far from straightforward, owing to three core issues: (1) entanglement in attribute-level semantics, (2) inconsistency across modalities, and (3) supervised signal missing. To tackle the above obstacles, we introduce a COMposed image retrieval network guided By attrIbute-based NEighbor Relations (COMBINER). Specifically, we first design an Adaptive Semantic Disentanglement module, which is capable of disentangling attribute features based on multimodal primitive features. Secondly, we propose a Unified Prototype-based Composition module, which can construct cross-modal unified prototypes (CUP) and facilitate multimodal feature composition. Finally, we introduce a Dual Relations Modeling module, which can mine pairwise and neighbor relations based on attribute similarity. Compared to traditional neighbor relations modeling CIR methods, COMBINER represents the first study addressing the phenomenon of visually similar but attribute-unrelated samples. It achieves a more accurate understanding of the semantic relations among samples by employing an attribute prototype-based similarity metric. Comprehensive experiments conducted on three benchmark datasets confirm the effectiveness of our proposed COMBINER. The implementation of our method will be accessed at https://github.com/Lee-zixu/COMBINER
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。