用小波编码压缩3D模型,实现百亿参数高效生成高精度3D形状。
Wavelet Latent Diffusion (Wala): Billion-Parameter 3D Generative Model with Compact Wavelet Encodings
- 用小波变换将256³体素距场压缩为12³×4的潜变量网格
- 实现2427倍压缩比,仍保留细节,支持百亿参数模型训练
- 生成速度2-4秒,开源最大多模态3D生成模型
大规模3D生成模型需大量计算资源,却难以在高分辨率下捕捉精细细节与复杂几何结构。我们归因于现有表示方式效率不足,缺乏建模所需的紧凑性。为此,提出新的小波潜扩散方法(WaLa),将3D形状编码为基于小波的紧凑潜变量。具体而言,将256³的符号距离场压缩为12³×4的潜变量网格,实现2427倍压缩比且损失极小。该高压缩率使模型可在不增加推理时间的前提下高效训练大规模生成网络。我们的条件与非条件模型均包含约十亿参数,成功在256³分辨率下生成高质量3D形状。此外,WaLa具备快速推理能力,根据条件不同可在2至4秒内生成形状。我们在多个数据集上实现业界领先性能,显著提升生成质量、多样性与计算效率。代码已开源,据我们所知,发布了跨模态最大的预训练3D生成模型。
原文摘要 · Abstract (English)
Large-scale 3D generative models require substantial computational resources yet often fall short in capturing fine details and complex geometries at high resolutions. We attribute this limitation to the inefficiency of current representations, which lack the compactness required to model the generative models effectively. To address this, we introduce a novel approach called Wavelet Latent Diffusion, or WaLa, that encodes 3D shapes into wavelet-based, compact latent encodings. Specifically, we compress a $256^3$ signed distance field into a $12^3 \times 4$ latent grid, achieving an impressive 2427x compression ratio with minimal loss of detail. This high level of compression allows our method to efficiently train large-scale generative networks without increasing the inference time. Our models, both conditional and unconditional, contain approximately one billion parameters and successfully generate high-quality 3D shapes at $256^3$ resolution. Moreover, WaLa offers rapid inference, producing shapes within two to four seconds depending on the condition, despite the model's scale. We demonstrate state-of-the-art performance across multiple datasets, with significant improvements in generation quality, diversity, and computational efficiency. We open-source our code and, to the best of our knowledge, release the largest pretrained 3D generative models across different modalities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。