将嵌入模型权重三值化,在不降性能前提下大幅降低内存与计算开销。
Efficient Ternary Weight Embedding Model: Bridging Scalability and Performance
- 采用自教知识蒸馏法实现预训练嵌入模型的三值化微调。
- 三值化后模型推理延迟低,内存占用少,精度几乎不变。
- 特别适合资源受限场景下的实时推荐系统部署。
嵌入模型已成为自然语言处理和计算机视觉中的关键工具,广泛用于语义搜索、推荐、聚类等任务。然而,全精度嵌入模型的高内存与计算需求限制了其在资源受限环境(如实时推荐系统)中的部署。本文提出一种新型微调框架,对嵌入模型进行三值化处理,在保持高性能的同时显著降低内存与计算开销。为将三值化应用于预训练嵌入模型,我们引入自教知识蒸馏方法,完成线性层的三值权重优化。在多个公开文本与视觉数据集上的大量实验表明,三值化模型在不损失有效性的情况下,具备极低的内存占用与推理延迟。在实际应用中,嵌入模型通常与近似最近邻(ANN)搜索结合使用,实验显示三值嵌入与ANN结合后,在准确率与计算效率上均取得显著提升。代码仓库已开源。
原文摘要 · Abstract (English)
Embedding models have become essential tools in both natural language processing and computer vision, enabling efficient semantic search, recommendation, clustering, and more. However, the high memory and computational demands of full-precision embeddings pose challenges for deployment in resource-constrained environments, such as real-time recommendation systems. In this work, we propose a novel finetuning framework to ternary-weight embedding models, which reduces memory and computational overhead while maintaining high performance. To apply ternarization to pre-trained embedding models, we introduce self-taught knowledge distillation to finalize the ternary-weights of the linear layers. With extensive experiments on public text and vision datasets, we demonstrated that without sacrificing effectiveness, the ternarized model consumes low memory usage and has low latency in the inference stage with great efficiency. In practical implementations, embedding models are typically integrated with Approximate Nearest Neighbor (ANN) search. Our experiments combining ternary embedding with ANN search yielded impressive improvement in both accuracy and computational efficiency. The repository is available at here.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。