通过反向去噪提升压缩概率估计精度,解决稀疏上下文下预测过平问题。
Micro-Diffusion Compression - Binary Tree Tweedie Denoising for Online Probability Estimation
- 将先验平滑视为收缩过程,用反向去噪校正概率分布
- 构建比特树结构,将256维校准拆为多层二元决策,数据高效
- 轻量级后处理模块,适配在线压缩系统,可修正系统性偏差
我们提出Midicoth,一种损失无损压缩系统,引入微扩散去噪层以改进自适应统计模型生成的概率估计。在如预测按部分匹配(PPM)的压缩器中,概率估计通过先验平滑处理稀疏观测。当上下文出现次数极少时,先验主导预测,导致分布显著平坦于真实源分布,造成压缩效率低下。Midicoth将先验平滑视为收缩过程,通过反向去噪步骤,利用经验校准统计修正预测概率。为提高数据效率,该方法将每个字节预测分解为沿比特树的层级二元决策,将单一256维校准问题转化为一系列二元校准任务,使少量观测即可实现可靠校准。去噪过程分多步进行,每阶段修正前一阶段遗留的残差误差。微扩散层作为轻量级后融合校准阶段,在所有模型预测合并后应用,可纠正最终概率分布中的系统性偏差。Midicoth整合五个完全在线组件:自适应PPM模型、长程匹配模型、基于字典树的词模型、高阶上下文模型,以及作为最终阶段的微扩散去噪器。
原文摘要 · Abstract (English)
We present Midicoth, a lossless compression system that introduces a micro-diffusion denoising layer for improving probability estimates produced by adaptive statistical models. In compressors such as Prediction by Partial Matching (PPM), probability estimates are smoothed by a prior to handle sparse observations. When contexts have been seen only a few times, this prior dominates the prediction and produces distributions that are significantly flatter than the true source distribution, leading to compression inefficiency. Midicoth addresses this limitation by treating prior smoothing as a shrinkage process and applying a reverse denoising step that corrects predicted probabilities using empirical calibration statistics. To make this correction data-efficient, the method decomposes each byte prediction into a hierarchy of binary decisions along a bitwise tree. This converts a single 256-way calibration problem into a sequence of binary calibration tasks, enabling reliable estimation of correction terms from relatively small numbers of observations. The denoising process is applied in multiple successive steps, allowing each stage to refine residual prediction errors left by the previous one. The micro-diffusion layer operates as a lightweight post-blend calibration stage applied after all model predictions have been combined, allowing it to correct systematic biases in the final probability distribution. Midicoth combines five fully online components: an adaptive PPM model, a long-range match model, a trie-based word model, a high-order context model, and the micro-diffusion denoiser applied as the final stage.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。