用注意力机制压缩人脸识别模型，单卡训练超大规模数据
AttFC: Attention Fully-Connected Layer for Large-Scale Face Recognition with One GPU
- 引入注意力全连接层，动态生成并存储类中心
- 参数量仅为传统全连接层的1/10，显著降低内存占用
- 适合资源受限场景，单张显卡即可训练百万级人脸数据
随着深度神经网络和大规模数据集的发展，人脸识别模型性能已大幅提升。然而，在大规模数据上训练时，全连接层参数量随身份数量线性增长，导致模型过大，对计算资源（如时间和内存）需求极高。本文提出注意力全连接（AttFC）层，通过注意力加载器生成生成式类别中心（GCC），并利用动态类别容器（DCC）仅存储部分类别中心，大幅减少参数量。该方法在单张显卡上训练大规模人脸识别模型时有效避免了显存溢出（OOM）问题，且性能与当前最优方法相当。
原文摘要 · Abstract (English)
Nowadays, with the advancement of deep neural networks (DNNs) and the availability of large-scale datasets, the face recognition (FR) model has achieved exceptional performance. However, since the parameter magnitude of the fully connected (FC) layer directly depends on the number of identities in the dataset. If training the FR model on large-scale datasets, the size of the model parameter will be excessively huge, leading to substantial demand for computational resources, such as time and memory. This paper proposes the attention fully connected (AttFC) layer, which could significantly reduce computational resources. AttFC employs an attention loader to generate the generative class center (GCC), and dynamically store the class center with Dynamic Class Container (DCC). DCC only stores a small subset of all class centers in FC, thus its parameter count is substantially less than the FC layer. Also, training face recognition models on large-scale datasets with one GPU often encounter out-of-memory (OOM) issues. AttFC overcomes this and achieves comparable performance to state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。