arXiv:2603.21233cs.CV2026-03

用物理启发的混合模型,实现高效深度图压缩。

DepthTCM: High Efficient Depth Compression via Physics-aware Transformer-CNN Mixed Architecture

  • 将深度图转为三通道图像,再用混合神经网络压缩
  • 0.307 bpp下保持99.38%精度,接近无损PNG效果
  • 4比特量化降低66%码率,适合移动端实时应用

我们提出DepthTCM,一种基于物理启发的端到端深度图压缩框架。该框架首先采用类正弦条纹投影测量技术,将高比特深度图无损转换为三通道图像表示;随后利用近期发展的Transformer-CNN混合神经网络对三通道图像进行编码与解码。具体而言,DepthTCM通过多波长深度(MWD)编码将深度映射为平滑三通道表示,全局量化至每通道4比特以降低熵,并最终使用结合卷积与Transformer层的自学习编解码器完成压缩。实验表明,该方法在Middlebury 2014数据集上达到0.307 bpp的同时保留99.38%精度,性能接近无损PNG。此外,我们在ScanNet++ iPhone RGB-D子集上报告了平均41.48毫秒(编码器)和47.45毫秒(解码器)的端到端推理时间,验证其高效性与可扩展性。消融实验显示,相较于8比特量化,4比特量化使码率降低66%,仅导致0.68 dB PSNR下降与0.04%精度损失;同时,Transformer-CNN模块相较纯CNN架构可提升最高0.75 dB PSNR。

原文摘要 · Abstract (English)

We propose DepthTCM, a physics-aware end-to-end framework for depth map compression. In our framework of DepthTCM, the high-bit depth map is first converted to a conventional 3-channel image representation losslessly using a method inspired by a physical sinusoidal fringe pattern based profiliometry system, then the 3-channel color image is encoded and decoded by a recently developed Transformer-CNN mixed neural network architecture. Specifically, DepthTCM maps depth to a smooth 3-channel using multiwavelength depth (MWD) encoding, then globally quantized the MWD encoded representation to 4 bits per channel to reduce entropy, and finally is compressed using a learned codec that combines convolutional and Transformer layers. Experiment results demonstrate the advantage of our proposed method. On Middlebury 2014, DepthTCM reaches 0.307 bpp while preserving 99.38% accuracy, a level of fidelity commensurate with lossless PNG. We additionally demonstrate practical efficiency and scalability, reporting average end-to-end inference times of 41.48 ms (encoder) and 47.45 ms (decoder) on the ScanNet++ iPhone RGB-D subset. Ablations validate our design choices: relative to 8-bit quantization, 4-bit quantization reduces bitrate by 66% while maintaining comparable reconstruction quality, with only a marginal 0.68 dB PSNR change and a 0.04% accuracy difference. In addition, Transformer--CNN blocks further improve PSNR by up to 0.75 dB over CNN-only architectures.

深度图压缩混合架构物理启发低码率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。