用渐进聚类优化提升视觉Transformer在人脸识别中的表现
LVFace: Progressive Cluster Optimization for Large Vision Models in Face Recognition
- 通过渐进聚类优化实现特征快速对齐与中心点稳定
- 在多个基准上超越UniFace和TopoFR，达新最优
- 适合大规模人脸数据与主流大模型融合应用
视觉变压器（ViTs）已革新大规模视觉建模，但在人脸识别（FR）领域仍受制于传统CNN训练范式，导致性能不足且收敛不稳定。本文提出基于ViT的人脸识别模型LVFace，引入渐进聚类优化（PCO），通过负类别子采样（NCS）实现从随机初始化的鲁棒快速特征对齐，结合特征期望惩罚保持中心点稳定，并在全批量训练中完成聚类边界精修。实验表明，LVFace在多个基准上超越UniFace和TopoFR等领先方法，建立新基准；其在大规模数据集上具可扩展性，兼容主流视觉语言模型（VLMs）与大语言模型（LLMs）。尤其在ICCV 2021 Masked Face Recognition（MFR）-Ongoing Challenge（2025年3月）中获得第一名，验证其在真实场景中的有效性。项目开源：https://github.com/bytedance/LVFace。
原文摘要 · Abstract (English)
Vision Transformers (ViTs) have revolutionized large-scale visual modeling, yet remain underexplored in face recognition (FR) where CNNs still dominate. We identify a critical bottleneck: CNN-inspired training paradigms fail to unlock ViT's potential, leading to suboptimal performance and convergence instability.To address this challenge, we propose LVFace, a ViT-based FR model that integrates Progressive Cluster Optimization (PCO) to achieve superior results. Specifically, PCO sequentially applies negative class sub-sampling (NCS) for robust and fast feature alignment from random initialization, feature expectation penalties for centroid stabilization, performing cluster boundary refinement through full-batch training without NCS constraints. LVFace establishes a new state-of-the-art face recognition baseline, surpassing leading approaches such as UniFace and TopoFR across multiple benchmarks. Extensive experiments demonstrate that LVFace delivers consistent performance gains, while exhibiting scalability to large-scale datasets and compatibility with mainstream VLMs and LLMs. Notably, LVFace secured 1st place in the ICCV 2021 Masked Face Recognition (MFR)-Ongoing Challenge (March 2025), proving its efficacy in real-world scenarios. Project is available at https://github.com/bytedance/LVFace.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。