用机器学习模型无损压缩有序整数键，提升数据库性能。
Learned Data Compression: Challenges and Opportunities for the Future
- 用分段线性模型近似有序键，结合残差数组实现无损压缩。
- 基于SIMD优化的模型在压缩效率上显著优于传统CPU压缩器。
- 适合关注数据库压缩、高效存储系统的研究人员和工程师。
在数据库管理（DB）、信息检索（IR）和高性能计算（HPC）等领域，压缩整数键是一项基础操作。近年来，学习型索引的进展激发了学习型压缩器的发展，这类压缩器利用简单而紧凑的机器学习（ML）模型对大规模有序键进行压缩。其核心思想是通过误差有界的ML模型（如分段线性函数）逼近有序键，并使用残差数组保证键的精确重建，实现无损压缩。尽管学习型压缩器仍处于早期探索阶段，我们的基准测试表明，经SIMD优化的学习型压缩器可显著超越当前主流的基于CPU的压缩器。本文基于初步实验，探讨学习型数据压缩在提升数据库管理系统（DBMS）及相关领域中的潜力，并指出现有系统在整合该新兴方法时需克服的关键技术挑战。
原文摘要 · Abstract (English)
Compressing integer keys is a fundamental operation among multiple communities, such as database management (DB), information retrieval (IR), and high-performance computing (HPC). Recent advances in \emph{learned indexes} have inspired the development of \emph{learned compressors}, which leverage simple yet compact machine learning (ML) models to compress large-scale sorted keys. The core idea behind learned compressors is to \emph{losslessly} encode sorted keys by approximating them with \emph{error-bounded} ML models (e.g., piecewise linear functions) and using a \emph{residual array} to guarantee accurate key reconstruction. While the concept of learned compressors remains in its early stages of exploration, our benchmark results demonstrate that an SIMD-optimized learned compressor can significantly outperform state-of-the-art CPU-based compressors. Drawing on our preliminary experiments, this vision paper explores the potential of learned data compression to enhance critical areas in DBMS and related domains. Furthermore, we outline the key technical challenges that existing systems must address when integrating this emerging methodology.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。