用预训练3D模型做超高压缩,1秒内还原高保真3D数据。
Squeeze3D: Your 3D Generation Model is Secretly an Extreme Neural Compressor
- 用可训练映射网络连接编码器与生成模型的隐空间
- 纹理网格压缩比达2187倍,点云619倍,保持视觉质量
- 无需真实3D数据训练,适配多种格式和现有模型
我们提出Squeeze3D,一种利用预训练3D生成模型隐式先验知识,在极高压缩比下压缩3D数据的新框架。通过可训练映射网络连接预训练编码器与生成模型的隐空间,任意以网格、点云或辐射场表示的3D模型均可被编码为高度紧凑的潜在代码,该代码可作为极压缩表示。映射网络将压缩后的潜在代码转换至强大生成模型的隐空间,再条件化重建原始3D模型(即解压)。Squeeze3D仅在合成数据上训练,不依赖任何真实3D数据集。该架构可灵活适配现有预训练编码器与生成模型,支持网格、点云和辐射场等多种格式。实验表明,其在纹理网格上实现最高2187倍压缩比,点云55倍,辐射场619倍,同时视觉质量优于多数现有方法。压缩与解压延迟极低,因无需为特定物体训练专属网络。
原文摘要 · Abstract (English)
We propose Squeeze3D, a novel framework that leverages implicit prior knowledge learnt by existing pre-trained 3D generative models to compress 3D data at extremely high compression ratios. Our approach bridges the latent spaces between a pre-trained encoder and a pre-trained generation model through trainable mapping networks. Any 3D model represented as a mesh, point cloud, or a radiance field is first encoded by the pre-trained encoder and then transformed (i.e. compressed) into a highly compact latent code. This latent code can effectively be used as an extremely compressed representation of the mesh or point cloud. A mapping network transforms the compressed latent code into the latent space of a powerful generative model, which is then conditioned to recreate the original 3D model (i.e. decompression). Squeeze3D is trained entirely on generated synthetic data and does not require any 3D datasets. The Squeeze3D architecture can be flexibly used with existing pre-trained 3D encoders and existing generative models. It can flexibly support different formats, including meshes, point clouds, and radiance fields. Our experiments demonstrate that Squeeze3D achieves compression ratios of up to 2187x for textured meshes, 55x for point clouds, and 619x for radiance fields while maintaining visual quality comparable to many existing methods. Squeeze3D only incurs a small compression and decompression latency since it does not involve training object-specific networks to compress an object.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。