提出新型压缩编码器，实现高效低损数据压缩，支持直接在压缩域学习。
Learned Compression for Compressed Learning
- 结合小波变换与非线性自编码器，实现线性压缩与降维的协同优化。
- 在图像分类等任务上性能超越主流扩散模型的编码器，保持高频细节。
- 全线性编码器设计适合移动端与远程传感，兼容多模态数据。
现代传感器生成越来越丰富的高分辨率数据流。受资源限制，机器学习系统通常通过降低分辨率丢弃大量信息。压缩域学习允许模型在紧凑的潜在表示上运行，从而在相同预算下实现更高有效分辨率。然而，现有压缩系统并不理想：线性变换编码和端到端学习压缩系统虽降低码率，但未均匀减少维度，无法显著提升效率；生成式自编码器虽降低维度，但其对抗性或感知目标导致严重信息损失。为此，我们提出 WaLLoC（小波学习型有损压缩），一种融合线性变换编码与非线性降维自编码器的神经编解码架构。WaLLoC 在可逆的小波包变换之间嵌入浅层、非对称自编码器与熵瓶颈。在多个关键指标上，WaLLoC 超越了当前先进潜在扩散模型所用的自编码器。其无需感知或对抗损失即可保留高频细节，适用于超出 RGB 图像和立体音频的模态。编码器几乎全部由线性操作构成，效率极高，适合移动计算、遥感及直接从压缩数据中学习。我们在图像分类、着色、文档理解、音乐源分离等多个任务上展示了 WaLLoC 在压缩域学习中的能力。代码、实验与预训练的音视频编解码器可在 https://ut-sysml.org/walloc 获取。
原文摘要 · Abstract (English)
Modern sensors produce increasingly rich streams of high-resolution data. Due to resource constraints, machine learning systems discard the vast majority of this information via resolution reduction. Compressed-domain learning allows models to operate on compact latent representations, allowing higher effective resolution for the same budget. However, existing compression systems are not ideal for compressed learning. Linear transform coding and end-to-end learned compression systems reduce bitrate, but do not uniformly reduce dimensionality; thus, they do not meaningfully increase efficiency. Generative autoencoders reduce dimensionality, but their adversarial or perceptual objectives lead to significant information loss. To address these limitations, we introduce WaLLoC (Wavelet Learned Lossy Compression), a neural codec architecture that combines linear transform coding with nonlinear dimensionality-reducing autoencoders. WaLLoC sandwiches a shallow, asymmetric autoencoder and entropy bottleneck between an invertible wavelet packet transform. Across several key metrics, WaLLoC outperforms the autoencoders used in state-of-the-art latent diffusion models. WaLLoC does not require perceptual or adversarial losses to represent high-frequency detail, providing compatibility with modalities beyond RGB images and stereo audio. WaLLoC's encoder consists almost entirely of linear operations, making it exceptionally efficient and suitable for mobile computing, remote sensing, and learning directly from compressed data. We demonstrate WaLLoC's capability for compressed-domain learning across several tasks, including image classification, colorization, document understanding, and music source separation. Our code, experiments, and pre-trained audio and image codecs are available at https://ut-sysml.org/walloc
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。