NeurStore通过细粒度张量存储与增量量化,显著降低深度模型在数据库中的存储开销。
NeurStore: Efficient In-database Deep Learning Model Management System
- 基于张量的分块存储,利用相似性阈值仅存差异数据,实现张量级去重。
- 提出增量量化算法,在保持精度可控前提下压缩率优于现有方法。
- 支持直接在压缩张量上计算,提升模型加载与使用效率,适合大规模模型管理场景。
随着数据库内AI分析的普及,对深度学习模型高效管理的需求日益增长。现有系统通常将模型作为整体文件存储或采用忽略模型结构特性的压缩技术,导致存储开销过大。本文提出NeurStore,一种新型数据库内模型管理系统,可高效存储与使用深度学习模型。首先,NeurStore采用基于张量的存储引擎,通过改进层次可导航小世界(HNSW)图来索引张量,并仅在预设相似性阈值内存储额外增量,实现张量级去重。其次,提出一种增量量化算法,有效压缩增量张量,实现更高压缩比且可控精度损失。最后,设计了压缩感知的模型加载机制,支持直接在压缩张量上进行计算,提升模型利用率。实验表明,相较于现有最佳方法,NeurStore在压缩比和模型加载吞吐量方面均表现更优。
原文摘要 · Abstract (English)
With the prevalence of in-database AI-powered analytics, there is an increasing demand for database systems to efficiently manage the ever-expanding number and size of deep learning models. However, existing database systems typically store entire models as monolithic files or apply compression techniques that overlook the structural characteristics of deep learning models, resulting in suboptimal model storage overhead. This paper presents NeurStore, a novel in-database model management system that enables efficient storage and utilization of deep learning models. First, NeurStore employs a tensor-based model storage engine to enable fine-grained model storage within databases. In particular, we enhance the hierarchical navigable small world (HNSW) graph to index tensors, and only store additional deltas for tensors within a predefined similarity threshold to ensure tensor-level deduplication. Second, we propose a delta quantization algorithm that effectively compresses delta tensors, thus achieving a superior compression ratio with controllable model accuracy loss. Finally, we devise a compression-aware model loading mechanism, which improves model utilization performance by enabling direct computation on compressed tensors. Experimental evaluations demonstrate that NeurStore achieves superior compression ratios and competitive model loading throughput compared to state-of-the-art approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。