arXiv:2507.00506cs.CV2025-07被引 2

通过动态提示融合与扰动一致性对齐,提升行人重识别效率与鲁棒性。

SCING:Towards More Efficient and Robust Person Re-Identification through Selective Cross-modal Prompt Tuning

  • 设计轻量级视觉提示融合模块,动态注入关键视觉特征到文本提示中。
  • 在多个基准上达到领先性能,如Market1501上mAP达87.6%,Rank-1达94.3%。
  • 无需复杂适配器,推理高效,适合部署于资源受限场景。

针对当前基于视觉-语言预训练模型(如CLIP)的行人重识别方法普遍依赖复杂适配器或模态特异性调优,忽视跨模态交互,导致计算开销大、对真实世界扰动不鲁棒的问题,本文提出一种名为选择性跨模态提示调优(SCING)的简单而高效框架。该方法引入两项创新:一是选择性视觉提示融合(SVIP),通过跨模态门控机制将判别性视觉特征动态注入文本提示;二是扰动驱动一致性对齐(PDCA),采用双路径训练策略,在随机图像扰动下强制原始与增强样本的跨模态嵌入保持一致。在Market1501、DukeMTMC-ReID、Occluded-Duke、Occluded-REID和P-DukeMTMC等多个主流数据集上的大量实验表明,该方法在不使用重型适配器的前提下,实现了高性能与低计算开销的平衡,显著提升跨模态对齐效果与鲁棒性。代码将在接受后发布。

原文摘要 · Abstract (English)

Recent advancements in adapting vision-language pre-training models like CLIP for person re-identification (ReID) tasks often rely on complex adapter design or modality-specific tuning while neglecting cross-modal interaction, leading to high computational costs or suboptimal alignment. To address these limitations, we propose a simple yet effective framework named Selective Cross-modal Prompt Tuning (SCING) that enhances cross-modal alignment and robustness against real-world perturbations. Our method introduces two key innovations: Firstly, we proposed Selective Visual Prompt Fusion (SVIP), a lightweight module that dynamically injects discriminative visual features into text prompts via a cross-modal gating mechanism. Moreover, the proposed Perturbation-Driven Consistency Alignment (PDCA) is a dual-path training strategy that enforces invariant feature alignment under random image perturbations by regularizing consistency between original and augmented cross-modal embeddings. Extensive experiments are conducted on several popular benchmarks covering Market1501, DukeMTMC-ReID, Occluded-Duke, Occluded-REID, and P-DukeMTMC, which demonstrate the impressive performance of the proposed method. Notably, our framework eliminates heavy adapters while maintaining efficient inference, achieving an optimal trade-off between performance and computational overhead. The code will be released upon acceptance.

行人重识别跨模态学习提示调优高效模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。