解决神经压缩中残差信号衰减问题，提升多阶段量化效果
Robust Residual Finite Scalar Quantization for Neural Compression
- 引入可学习缩放因子与可逆层归一化，稳定多阶段残差信号
- 音频重建提升3.6%，图像重建L1损失降低9.7%、感知损失降17.4%
- 适合追求高效高保真压缩的开发者，尤其在音频与图像领域
有限标量量化（FSQ）虽简化了训练过程，但在多阶段设置中面临残差幅值衰减问题，后续阶段接收到的信号呈指数级减弱。本文提出鲁棒残差有限标量量化（RFSQ），通过两个新设计的条件策略——可学习缩放因子和可逆层归一化——解决这一根本缺陷。跨音频与图像模态的实验表明，RFSQ具有显著有效性与泛化能力。在24 bit/frame音频重建中，RFSQ-LayerNorm达到3.646 DNSMOS，较最优基线RVQ（3.518）提升3.6%。在ImageNet上，RFSQ实现0.102 L1损失与0.100感知损失，其中LayerNorm策略相比无条件变体分别带来9.7%的L1改进与17.4%的感知改进。该策略通过保持各阶段输入统计量的归一化，有效防止了朴素残差方法中的指数级幅值衰减。RFSQ将FSQ的简洁性与多阶段量化表达力结合，为跨模态神经压缩树立了新标准。
原文摘要 · Abstract (English)
Finite Scalar Quantization (FSQ) offers simplified training but suffers from residual magnitude decay in multi-stage settings, where subsequent stages receive exponentially weaker signals. We propose Robust Residual Finite Scalar Quantization (RFSQ), addressing this fundamental limitation through two novel conditioning strategies: learnable scaling factors and invertible layer normalization. Our experiments across audio and image modalities demonstrate RFSQ's effectiveness and generalizability. In audio reconstruction at 24 bits/frame, RFSQ-LayerNorm achieves 3.646 DNSMOS, a 3.6% improvement over state-of-the-art RVQ (3.518). On ImageNet, RFSQ achieves 0.102 L1 loss and 0.100 perceptual loss, with LayerNorm providing 9.7% L1 improvement and 17.4% perceptual improvement over unconditioned variants. The LayerNorm strategy consistently outperforms alternatives by maintaining normalized input statistics across stages, effectively preventing exponential magnitude decay that limits naive residual approaches. RFSQ combines FSQ's simplicity with multi-stage quantization's representational power, establishing a new standard for neural compression across diverse modalities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。