提出PRISM框架,提升多模态物体重识别的精度与效率
Multi-Modal Object Re-Identification with Prompt-S6 and Semantic-Aware Knowledge Guidance

- 基于Prompt-S6实现跨模态高效交互,保持线性复杂度
- 通过语义驱动的动态裁剪抑制背景干扰,提升特征质量
- 渐进式融合网络实现三模态对齐,适合复杂场景应用
多模态物体重识别(ReID)旨在通过融合多种模态的互补信息来检索特定物体。然而,现有方法未能有效抑制背景干扰或实现三模态对齐,多数聚焦于成对特征融合,且聚合方式计算开销高。为此,本文提出PRISM框架,基于Prompt-S6(PS6)和语义感知知识引导。PS6在保持Mamba线性复杂度和强序列建模能力的同时,支持高效的跨模态交互。在此基础上,设计两个关键组件:语义驱动的令牌裁剪(SDTP)与渐进式融合网络(PFN)。SDTP从分割基础模型中解析语义先验,利用其进行动态令牌裁剪,以抑制背景噪声并优化特征表示;PFN逐步聚合多模态特征,实现三模态对齐,充分挖掘模态互补性。所提方法在四个多模态物体ReID基准上验证了有效性与高效性。代码已开源。
原文摘要 · Abstract (English)
Multi-modal object Re-Identification (ReID) aims to retrieve specific objects by integrating complementary information from multiple modalities. However, existing multi-modal ReID methods do not effectively address background interference suppression or achieve tri-modal alignment, instead focusing on pairwise feature fusion. Moreover, many current aggregation approaches suffer from high computational complexity. To address these limitations, we propose PRISM, a novel multi-modal ReID framework built upon Prompt-S6 (PS6) and semantic-aware knowledge guidance. PS6 maintains the linear complexity and strong sequence modeling capability of Mamba while enabling efficient cross-modal interaction. Leveraging these advantages, we design two key components: Semantic-Driven Token Pruning (SDTP) and Progressive Fusion Network (PFN). Parsing semantic priors from the segmentation foundation models, the SDTP then leverages these priors and applies dynamic token pruning to suppress background noise and refine feature representations. The PFN progressively aggregates multi-modal features to achieve tri-modal alignment and fully exploit modality complementarity. With the proposed modules, PRISM generates more robust multi-modal representations under complex scenarios. Extensive experiments on four multi-modal object ReID benchmarks demonstrate the effectiveness and efficiency of our approach. The source code is available at https://github.com/zw-absin/PRISM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。