通过生成外观先验对齐,提升细粒度图像检索对未见类别的泛化能力。
Learning to Align Generative Appearance Priors for Fine-grained Image Retrieval

- 用可逆流模型建模类别外观先验,实现特征空间到密度空间的双向映射。
- 在未见类别上达到最新最佳性能,显著提升跨类别泛化能力。
- 适合做细粒度图像检索、跨类别泛化研究的学者参考。
细粒度图像检索(FGIR)通常依赖已见类别的监督信号来学习判别性嵌入,以检索未见类别。然而,这种监督常使模型偏向已见类别的语义,而非具有跨类别泛化性的底层外观特征,从而限制了在未见类别上的检索性能。为此,我们提出GAPan——一种生成外观先验对齐网络,将学习目标从类别预测重构为外观建模。技术上,GAPan采用基于归一化流的可逆密度模型处理检索特征。前向传播中,流将所有实例特征映射至潜密度空间,每个已见类别由条件高斯先验建模,并通过精确似然估计优化。该设定利用流的可逆性保留更丰富的外观细节。反向传播中,从这些学习先验的高密度区域采样,映射回特征空间生成反映类别内变化的外观感知锚点。这些锚点用于指导先验驱动的对齐目标,使检索嵌入与类别特异性外观分布对齐,从而提升对未见类别的泛化能力。实验表明,GAPan在多个主流细粒度和粗粒度基准上均达到当前最优性能。
原文摘要 · Abstract (English)
Fine-grained image retrieval (FGIR) typically relies on supervision from seen categories to learn discriminative embeddings for retrieving unseen categories. However, such supervision often biases retrieval models toward the semantics of seen categories rather than the underlying appearance characteristics that generalize across categories, thereby limiting retrieval performance on unseen categories. To tackle this, we propose GAPan, a Generative Appearance Prior alignment network that reformulates the learning objective from category prediction toward appearance modeling. Technically, GAPan treats retrieval features with an invertible density model based on normalizing flows. In the forward direction, the flow maps all instance features into a latent density space, where each seen category is modeled by a class-conditional Gaussian prior and optimized via exact likelihood estimation. This formulation preserves richer appearance details by leveraging the invertible property of the flows. In the reverse direction, samples from the high-density regions of these learned priors are mapped back to the feature space to produce appearance-aware anchors that reflect intra-category variation. These anchors supervise a prior-driven alignment objective that aligns retrieval embeddings with category-specific appearance distributions, thereby improving generalization to unseen categories. Evaluations demonstrate that our GAPan achieves state-of-the-art performance on both widely-used fine- and coarse-grained benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。