自动分配特征维度，节省内存且效果不降。
DimGrow: Memory-Efficient Field-level Embedding Dimension Search
- 从每维一维开始，按重要性动态调整维度大小。
- 在三个推荐数据集上表现优于传统方法，内存降低显著。
- 适合大规模推荐系统中嵌入维度优化场景。
关键特征字段需要更大的嵌入维度，其他字段则需更小。这要求自动化维度分配。现有方法如剪枝或神经架构搜索（NAS）需训练一个内存密集的超网络（SuperNet），枚举所有可能的维度组合，对大型特征空间不可行。我们提出 DimGrow，一种轻量级方法，无需超网络。从每个特征字段初始化为一维开始，通过重要性评分逐步扩展或收缩维度。仅当重要性持续超过阈值时才增加维度，确保内存效率。在三个推荐数据集上的实验验证了 DimGrow 的有效性，同时相比基于超网络的方法显著降低训练内存消耗。
原文摘要 · Abstract (English)
Key feature fields need bigger embedding dimensionality, others need smaller. This demands automated dimension allocation. Existing approaches, such as pruning or Neural Architecture Search (NAS), require training a memory-intensive SuperNet that enumerates all possible dimension combinations, which is infeasible for large feature spaces. We propose DimGrow, a lightweight approach that eliminates the SuperNet requirement. Starting training model from one dimension per feature field, DimGrow can progressively expand/shrink dimensions via importance scoring. Dimensions grow only when their importance consistently exceed a threshold, ensuring memory efficiency. Experiments on three recommendation datasets verify the effectiveness of DimGrow while it reduces training memory compared to SuperNet-based methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。