用预定义向量系统加速超百万类神经网络分类，提速最高达11.6倍。
Using predefined vector systems to speed up neural network multimillion class classification
- 利用已知隐空间几何特性，将分类转为快速聚类中心搜索。
- 推理速度提升最高11.6倍，且不损失模型准确率。
- 支持新类别预测，适合大规模开放词汇分类场景。
神经网络中的标签预测复杂度与类别数呈O(n)关系，无论使用全连接层还是基于类原型的余弦相似度。本文表明，若神经网络隐空间（LS）几何结构具备特定性质，可通过将其配置到预定义的向量系统中，将标签预测复杂度降至O(1)。该方法仅需找出嵌入向量中若干最大和最小值的索引，计算极为高效。实验在多个数据集上验证了该方法不改变训练准确率，且显著降低推理各阶段耗时。结果表明，相比传统方法，整体推理速度最高可提升11.6倍。此外，该方法具备预测新类别存在的独特能力。
原文摘要 · Abstract (English)
Label prediction in neural networks (NNs) has O(n) complexity proportional to the number of classes. This holds true for classification using fully connected layers and cosine similarity with some set of class prototypes. In this paper we show that if NN latent space (LS) geometry is known and possesses specific properties, label prediction complexity can be significantly reduced. This is achieved by associating label prediction with the O(1) complexity closest cluster center search in a vector system used as target for latent space configuration (LSC). The proposed method only requires finding indexes of several largest and lowest values in the embedding vector making it extremely computationally efficient. We show that the proposed method does not change NN training accuracy computational results. We also measure the time required by different computational stages of NN inference and label prediction on multiple datasets. The experiments show that the proposed method allows to achieve up to 11.6 times overall acceleration over conventional methods. Furthermore, the proposed method has unique properties which allow to predict the existence of new classes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。