arXiv:2411.06023cs.CV2024-11被引 1

用动态文本提示实现无记忆持续行人重识别,避免遗忘与隐私问题。

Dynamic Textual Prompt For Rehearsal-free Lifelong Person Re-identification

  • 设计动态文本提示框架,用自然语言引导模型跨域学习统一特征空间。
  • 在多个数据集上超越当前最优方法,尤其在未见场景下性能更稳定。
  • 适合需要长期更新且受限于存储或隐私的实时监控系统使用。

持续行人重识别旨在跨摄像头识别人员并从连续数据流中融入新知识。核心挑战包括参数更新引发的灾难性遗忘和领域偏移,以及对已见与未见领域的性能保持。以往方法依赖数据记忆保留历史样本,但存储量随训练域数量线性增长,且在隐私限制下无法保存数据时性能显著下降。为此,本文提出以文本描述作为引导,促使模型在不保留样本的前提下学习跨域不变特征。关键洞察是自然语言能以一致风格描述行人实例,形成共享的文本空间。通过该空间作为锚点,可引导模型将不同域图像嵌入统一语义空间,缓解因领域变化导致的遗忘。为此,我们引入任务驱动的动态文本提示框架,包含动态提示融合模块,自适应构建并融合两种文本提示作为锚点;设计文本-视觉特征对齐模块,学习细粒度视觉与文本特征间的精确映射;还开发可学习的知识蒸馏模块,动态平衡旧知识保留与新知识获取。大量实验表明,该方法在多种设置下均优于现有最优方法。

原文摘要 · Abstract (English)

Lifelong person re-identification attempts to recognize people across cameras and integrate new knowledge from continuous data streams. Key challenges involve addressing catastrophic forgetting caused by parameter updating and domain shift, and maintaining performance in seen and unseen domains. Many previous works rely on data memories to retain prior samples. However, the amount of retained data increases linearly with the number of training domains, leading to continually increasing memory consumption. Additionally, these methods may suffer significant performance degradation when data preservation is prohibited due to privacy concerns. To address these limitations, we propose using textual descriptions as guidance to encourage the ReID model to learn cross-domain invariant features without retaining samples. The key insight is that natural language can describe pedestrian instances with an invariant style, suggesting a shared textual space for any pedestrian images. By leveraging this shared textual space as an anchor, we can prompt the ReID model to embed images from various domains into a unified semantic space, thereby alleviating catastrophic forgetting caused by domain shifts. To achieve this, we introduce a task-driven dynamic textual prompt framework in this paper. This model features a dynamic prompt fusion module, which adaptively constructs and fuses two different textual prompts as anchors. This effectively guides the ReID model to embed images into a unified semantic space. Additionally, we design a text-visual feature alignment module to learn a more precise mapping between fine-grained visual and textual features. We also developed a learnable knowledge distillation module that allows our model to dynamically balance retaining existing knowledge with acquiring new knowledge. Extensive experiments demonstrate that our method outperforms SOTAs under various settings.

持续学习行人重识别文本提示免记忆

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。