用频域建模提升图像压缩效率,保持重建确定性。
FLM: Frequency-Aware Language Models for Generative Image Compression

- 基于离散余弦系数的序列化建模,实现频域自回归预测。
- 在Kodak等数据集上比JPEG提升3.3~3.8 dB的BD-PSNR。
- 兼容有损与无损压缩,适合追求高保真和低伪影的应用。
生成模型通过利用学习到的先验知识,显著提升了低比特率下的图像有损压缩性能。然而,生成的纹理和语义细节可能偏离源内容,影响重建保真度。为此,我们提出FLM,一种频率感知的语言模型,通过频域概率建模提升压缩效率,同时保持确定性重建。编码器将输入图像转换为量化后的DCT系数,并通过宏块级系数分词构建离散序列。FLM执行逐系数预测,自回归地估计用于算术编码的条件概率分布,生成紧凑码流。解码器通过大语言模型与算术解码器联合恢复频域数据,再进行逆变换重构图像。我们构建了专用频域数据集并采用两阶段微调策略,使模型适用于多种比特率设置。FLM是一种通用压缩器,兼容有损与无损JPEG重压缩框架。实验表明,FLM在率失真性能上超越传统及生成式有损压缩方法,在Kodak、Tecnick、CLIC2020数据集上分别获得3.30 dB、3.83 dB、3.80 dB的BD-PSNR增益。定性结果显示,FLM在提升语义保真度、抑制块效应方面表现更优。此外,其在无损重压缩任务中也表现出色。
原文摘要 · Abstract (English)
Generative models have significantly improved the performance ceiling of image lossy compression at low bitrates by exploiting learned priors. However, the generated textures and semantic details may deviate from the source content, thereby affecting the fidelity of image reconstruction. To solve these challenges, we propose FLM, a frequency-aware language model that improves compression efficiency through frequency-domain probabilistic modeling while retaining deterministic reconstruction. At the encoder, the input image is transformed into quantized DCT coefficients, which are organized into discrete sequences using macroblock-based coefficient tokenization. FLM then performs next-coefficient prediction to autoregressively estimate token-wise conditional probability distributions for arithmetic coding, thereby generating a compact bitstream. At the decoder, the LLM and arithmetic decoder jointly recover the frequency-domain data, followed by inverse transformations for image reconstruction. A task-specific frequency-domain dataset and a two-stage fine-tuning strategy are further developed to enable the model to operate across multiple bitrate settings. FLM is a versatile compressor that is compatible with both lossy compression and lossless JPEG recompression frameworks. Experiments show that FLM exceeds conventional and generative lossy compression methods in rate-distortion performance. FLM achieves BD-PSNR gains of 3.30 dB, 3.83 dB, and 3.80 dB than JPEG baseline on Kodak, Tecnick, and CLIC2020, respectively. Better qualitative quality of FLM can be achieved in improving semantically high fidelity and suppressing blocking artifacts. FLM is also validated to be applicable to the lossless recompression task with competitive performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。