统一图像与文本行人重识别,用分层提示提升跨模态对齐
Hierarchical Prompt Learning for Image- and Text-Based Person Re-Identification
- 设计任务路由的Transformer,分离图像与文本分支特征
- 生成身份级与实例级伪文本提示,注入细粒度语义信息
- 跨模态提示正则化确保语义一致性,适合多模态检索场景
行人重识别(ReID)旨在根据视觉查询(图像到图像,I2I)或文本描述(文本到图像,T2I)检索目标行人图像。尽管两项任务共享相同检索目标,但面临不同挑战:I2I强调身份判别能力,而T2I需要精确的跨模态语义对齐。现有方法通常分别处理,易导致表征纠缠、性能不佳。为此,我们提出统一框架——分层提示学习(HPL),通过任务感知提示建模联合优化两项任务。首先引入任务路由Transformer,将双分类标记嵌入共享视觉编码器,分别引导特征至I2I和T2I分支。在此基础上,设计分层提示生成机制,融合身份级可学习标记与实例级伪文本标记。这些伪标记由图像或文本特征经模态专用反演网络生成,将细粒度实例语义注入提示。此外,提出跨模态提示正则化策略,在提示空间强制语义对齐,确保伪提示保留源模态特性并增强跨模态迁移性。在多个ReID基准上的大量实验验证了方法有效性,在I2I与T2I任务上均达到最先进水平。
原文摘要 · Abstract (English)
Person re-identification (ReID) aims to retrieve target pedestrian images given either visual queries (image-to-image, I2I) or textual descriptions (text-to-image, T2I). Although both tasks share a common retrieval objective, they pose distinct challenges: I2I emphasizes discriminative identity learning, while T2I requires accurate cross-modal semantic alignment. Existing methods often treat these tasks separately, which may lead to representation entanglement and suboptimal performance. To address this, we propose a unified framework named Hierarchical Prompt Learning (HPL), which leverages task-aware prompt modeling to jointly optimize both tasks. Specifically, we first introduce a Task-Routed Transformer, which incorporates dual classification tokens into a shared visual encoder to route features for I2I and T2I branches respectively. On top of this, we develop a hierarchical prompt generation scheme that integrates identity-level learnable tokens with instance-level pseudo-text tokens. These pseudo-tokens are derived from image or text features via modality-specific inversion networks, injecting fine-grained, instance-specific semantics into the prompts. Furthermore, we propose a Cross-Modal Prompt Regularization strategy to enforce semantic alignment in the prompt token space, ensuring that pseudo-prompts preserve source-modality characteristics while enhancing cross-modal transferability. Extensive experiments on multiple ReID benchmarks validate the effectiveness of our method, achieving state-of-the-art performance on both I2I and T2I tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。