用预设向量系统加速神经网络训练，支持超大规模分类。
Exploring possible vector systems for faster training of neural networks with preconfigured latent spaces
- 设计可配置的向量系统，直接定义隐空间结构以替代分类层。
- 在ImageNet-1K及60万类数据上实现训练速度显著提升。
- 最少维度配置可加速收敛，降低嵌入向量存储开销。
神经网络性能与其在隐空间中的嵌入分布特性密切相关。近期研究表明，使用特定的根系向量（An根系向量）作为隐空间配置（LSC）的目标，可确保期望的隐空间结构。该方法的核心优势在于无需分类层即可训练分类器神经网络，从而支持极大规模类别数据集的训练。本文系统探讨了适用于神经网络训练的多种向量系统及其构造方法，揭示其性质与适用性。这些系统被用于配置编码器与视觉变压器的隐空间，在ImageNet-1K及50万至60万类数据集上显著加速训练。同时发现，针对特定类别数采用最小隐空间维度可加快收敛速度，具有减少嵌入向量数据库规模的潜力。
原文摘要 · Abstract (English)
The overall neural network (NN) performance is closely related to the properties of its embedding distribution in latent space (LS). It has recently been shown that predefined vector systems, specifically An root system vectors, can be used as targets for latent space configurations (LSC) to ensure the desired LS structure. One of the main LSC advantage is the possibility of training classifier NNs without classification layers, which facilitates training NNs on datasets with extremely large numbers of classes. This paper provides a more general overview of possible vector systems for NN training along with their properties and methods for vector system construction. These systems are used to configure LS of encoders and visual transformers to significantly speed up ImageNet-1K and 50k-600k classes LSC training. It is also shown that using the minimum number of LS dimensions for a specific number of classes results in faster convergence. The latter has potential advantages for reducing the size of vector databases used to store NN embeddings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。