基于概念匹配，从图像中找共通叙事的相似图。
Concept Retrieval -- What and How?
- 用双峰高斯建模邻域结构，识别共享概念
- 在多个数据集上显著优于传统方法
- 适合需要理解图像深层语义的场景
一个概念可反映具体或抽象的思想。给定一张输入图像，本文旨在检索与其核心概念一致的其他图像，捕捉潜在叙述内容。这超越了传统检索或聚类方法对视觉或语义相似性的依赖。我们正式定义该问题，提出关键需求与评估指标。提出新方法，基于两个关键观察：(1) 嵌入空间中每个邻居通常与查询共享至少一个概念，但彼此间未必共享同一概念；(2) 用双峰高斯分布建模邻域结构，能揭示有意义的模式以辅助概念识别。定性、定量及人工评估均验证方法有效性。相关代码可在 PyPI 获取：https://pypi.org/project/coret/
原文摘要 · Abstract (English)
A concept may reflect either a concrete or abstract idea. Given an input image, this paper seeks to retrieve other images that share its central concepts, capturing aspects of the underlying narrative. This goes beyond conventional retrieval or clustering methods, which emphasize visual or semantic similarity. We formally define the problem, outline key requirements, and introduce appropriate evaluation metrics. We propose a novel approach grounded in two key observations: (1) While each neighbor in the embedding space typically shares at least one concept with the query, not all neighbors necessarily share the same concept with one another. (2) Modeling this neighborhood with a bimodal Gaussian distribution uncovers meaningful structure that facilitates concept identification. Qualitative, quantitative, and human evaluations confirm the effectiveness of our approach. See the package on PyPI: https://pypi.org/project/coret/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。