arXiv:2503.13739cs.CV2025-03CVPR被引 8

无需标注和相机参数,通过同步性自监督实现多视角行人关联

Learning from Synchronization: Self-Supervised Uncalibrated Multi-View Person Association in Challenging Scenes

  • 利用跨视角图像时间同步作为自监督信号,学习统一几何与外观特征
  • 在三个挑战性数据集上超越现有无监督与全监督方法,最高提升12.3%
  • 适合缺乏标注或相机标定信息的复杂监控场景应用

多视角行人关联是人体行为多视角分析的基础步骤。尽管行人重识别特征已被证明有效,但在外观相似的挑战性场景中不可靠。因此需要跨视角几何约束以增强鲁棒性。然而,现有方法大多依赖真实身份标签(全监督)或需已校准的相机参数,而后者难以获取。本文探索从同步性中学习的潜力,提出无需任何标注的自监督非校准多视角行人关联方法 Self-MVA。具体地,设计一个包含编码器-解码器结构和自监督预训练任务(跨视角图像同步)的框架,旨在判断不同视角图像是否同步。模型联合编码行人的几何与外观特征,并通过匈牙利匹配将实例级与图像级距离对齐后,利用同步标签进行监督训练。为进一步缩小解空间,引入两类自监督线性约束:多视角重投影与成对边关联。在三个公开挑战性数据集(WILDTRACK、MVOR、SOLDIERS)上的大量实验表明,本方法达到当前最优性能,优于现有无监督与全监督方法。代码已开源:https://github.com/CAMMA-public/Self-MVA。

原文摘要 · Abstract (English)

Multi-view person association is a fundamental step towards multi-view analysis of human activities. Although the person re-identification features have been proven effective, they become unreliable in challenging scenes where persons share similar appearances. Therefore, cross-view geometric constraints are required for a more robust association. However, most existing approaches are either fully-supervised using ground-truth identity labels or require calibrated camera parameters that are hard to obtain. In this work, we investigate the potential of learning from synchronization, and propose a self-supervised uncalibrated multi-view person association approach, Self-MVA, without using any annotations. Specifically, we propose a self-supervised learning framework, consisting of an encoder-decoder model and a self-supervised pretext task, cross-view image synchronization, which aims to distinguish whether two images from different views are captured at the same time. The model encodes each person's unified geometric and appearance features, and we train it by utilizing synchronization labels for supervision after applying Hungarian matching to bridge the gap between instance-wise and image-wise distances. To further reduce the solution space, we propose two types of self-supervised linear constraints: multi-view re-projection and pairwise edge association. Extensive experiments on three challenging public benchmark datasets (WILDTRACK, MVOR, and SOLDIERS) show that our approach achieves state-of-the-art results, surpassing existing unsupervised and fully-supervised approaches. Code is available at https://github.com/CAMMA-public/Self-MVA.

多视角关联自监督学习行人重识别无监督方法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。