用大模型预测下一个像素,实现高效无损图像压缩
Large Language Models for Lossless Image Compression: Next-Pixel Prediction in Language Space is All You Need
- 基于像素序列的下一像素预测,融合语义与上下文理解
- 在标准数据集上超越现有最先进无损压缩算法
- 适合对图像压缩效率有极致要求的研究与工程场景
近期研究表明,大语言模型(LLM)所具备的‘智能’与‘压缩’本质上是同一事物的两面,其强大的通用性使其成为多种数据模态的无损压缩器。这一特性尤其契合当前流媒体时代对高分辨率图像无损压缩的需求。然而,我们发现直接应用基于LLM的无损压缩方法在常见基准数据集上的性能仍显著落后于现有最先进的(SOTA)编码器。为此,我们致力于挖掘LLM在无损图像压缩任务中的潜在能力,以弥合理论与实际性能之间的差距。具体而言,我们提出P²-LLM,一种基于下一像素预测的LLM架构,融合了像素级先验知识、LLM的上下文学习能力以及像素级语义保持策略,增强对像素序列的理解能力,从而提升下一像素预测精度。大量实验表明,P²-LLM在多个基准数据集上优于现有的经典及学习型编码器。
原文摘要 · Abstract (English)
We have recently witnessed that ``Intelligence" and `` Compression" are the two sides of the same coin, where the language large model (LLM) with unprecedented intelligence is a general-purpose lossless compressor for various data modalities. This attribute particularly appeals to the lossless image compression community, given the increasing need to compress high-resolution images in the current streaming media era. Consequently, a spontaneous envision emerges: Can the compression performance of the LLM elevate lossless image compression to new heights? However, our findings indicate that the naive application of LLM-based lossless image compressors suffers from a considerable performance gap compared with existing state-of-the-art (SOTA) codecs on common benchmark datasets. In light of this, we are dedicated to fulfilling the unprecedented intelligence (compression) capacity of the LLM for lossless image compression tasks, thereby bridging the gap between theoretical and practical compression performance. Specifically, we propose P$^{2}$-LLM, a next-pixel prediction-based LLM, which integrates various elaborated insights and methodologies, \textit{e.g.,} pixel-level priors, the in-context ability of LLM, and a pixel-level semantic preservation strategy, to enhance the understanding capacity of pixel sequences for better next-pixel predictions. Extensive experiments on benchmark datasets demonstrate that P$^{2}$-LLM can beat SOTA classical and learned codecs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。