用深度优先搜索挖掘难样本，提升CLIP在跨域行人重识别中的表现。
CLIP-DFGS: A Hard Sample Mining Method for CLIP in Generalizable Person Re-Identification
- 基于深度优先搜索构建图采样器，自动筛选高难度对比样本。
- 在CUHK03和Market-1501上实现显著性能提升，验证方法有效性。
- 适用于图像与文本双编码器，增强模型对细微特征的区分能力。
预训练视觉语言模型如CLIP在行人重识别（ReID）中展现出潜力，但在通用场景下的表现仍不理想。其大规模、多样化的图文对预训练可能导致某些细粒度特征缺失或不足。为此，我们提出一种基于深度优先搜索的硬样本挖掘方法DFGS（Depth-First Graph Sampler），旨在为CLIP提供足够挑战性的样本，以增强其提取细粒度特征的能力。DFGS可应用于CLIP的图像编码器与文本编码器。通过利用CLIP强大的跨模态学习能力，我们使用DFGS挖掘具有高判别难度的样本并形成小批量训练数据，使图像模型能更高效地学习难以区分的样本，从而提升个体间的区分能力。实验结果表明，该方法在通用行人重识别任务中显著优于现有方法，证实了DFGS在提升模型性能方面的有效性。
原文摘要 · Abstract (English)
Recent advancements in pre-trained vision-language models like CLIP have shown promise in person re-identification (ReID) applications. However, their performance in generalizable person re-identification tasks remains suboptimal. The large-scale and diverse image-text pairs used in CLIP's pre-training may lead to a lack or insufficiency of certain fine-grained features. In light of these challenges, we propose a hard sample mining method called DFGS (Depth-First Graph Sampler), based on depth-first search, designed to offer sufficiently challenging samples to enhance CLIP's ability to extract fine-grained features. DFGS can be applied to both the image encoder and the text encoder in CLIP. By leveraging the powerful cross-modal learning capabilities of CLIP, we aim to apply our DFGS method to extract challenging samples and form mini-batches with high discriminative difficulty, providing the image model with more efficient and challenging samples that are difficult to distinguish, thereby enhancing the model's ability to differentiate between individuals. Our results demonstrate significant improvements over other methods, confirming the effectiveness of DFGS in providing challenging samples that enhance CLIP's performance in generalizable person re-identification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。