arXiv:2502.15934cs.CV2025-02被引 4

分析人体识别模型的嵌入表示,发现其隐含人脸、性别等信息

Dissecting Human Body Representations in Deep Networks Trained for Person Identification

  • 用四种架构分析190万图像的嵌入表示,揭示隐藏特征
  • 无显式训练也能识别人脸,且能区分性别与拍摄视角
  • 通过主成分分析压缩嵌入空间,可不训练提升识别准确率

近年来,随着高质量训练数据的增加,长期人体重识别算法迅速发展。本文通过对4个基于190万张图像(覆盖4788个身份,9个数据集)训练的人体识别网络进行分析,探索其嵌入表示的特性。研究涵盖ViT、SWIN-ViT、CNN及语言引导的CNN等多种架构。结果表明:1)面部信息对识别准确率有贡献,且模型能在未显式训练人脸的情况下实现一定程度的脸部识别;2)嵌入向量编码了性别、视角(yaw)甚至原始数据集等图像属性信息;3)无需额外训练,仅通过主成分分析(PCA)剔除方差大的维度,即可在子空间中实现更精准的身份比对。上述发现跨架构与测试数据集高度一致,首次系统揭示了长时重识别网络在复杂非受限数据下的表征特性。

原文摘要 · Abstract (English)

Long-term body identification algorithms have emerged recently with the increased availability of high-quality training data. We seek to fill knowledge gaps about these models by analyzing body image embeddings from four body identification networks trained with 1.9 million images across 4,788 identities and 9 databases. By analyzing a diverse range of architectures (ViT, SWIN-ViT, CNN, and linguistically primed CNN), we first show that the face contributes to the accuracy of body identification algorithms and that these algorithms can identify faces to some extent -- with no explicit face training. Second, we show that representations (embeddings) generated by body identification algorithms encode information about gender, as well as image-based information including view (yaw) and even the dataset from which the image originated. Third, we demonstrate that identification accuracy can be improved without additional training by operating directly and selectively on the learned embedding space. Leveraging principal component analysis (PCA), identity comparisons were consistently more accurate in subspaces that eliminated dimensions that explained large amounts of variance. These three findings were surprisingly consistent across architectures and test datasets. This work represents the first analysis of body representations produced by long-term re-identification networks trained on challenging unconstrained datasets.

人体识别嵌入分析特征解耦

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。