arXiv:2411.19037cs.CV2024-11被引 1

用小波分层预测生成高保真3D形状,速度更快、细节更准。

3D-WAG: Hierarchical Wavelet-Guided Autoregressive Generation for High-Fidelity 3D Shapes

  • 将3D形状转为多尺度小波令牌图,按层级逐步生成更高分辨率
  • 在覆盖度和分布匹配上优于现有方法,生成形状更接近真实数据
  • 适合需要高效可控生成的3D内容创作场景

自回归(AR)模型在自然语言和图像生成中表现卓越,但在3D形状建模中仍鲜有探索。与扩散模型相比,AR模型具有更高的生成效率和更强的可控性,特别适用于数据密集型任务。传统3D AR模型通常在体素或点级别进行‘下一个标记’预测,虽有效但对大规模3D数据存在计算开销大、限制多的问题。为此,我们提出3D-WAG,一种基于隐式距离场的3D自回归生成模型,支持无条件、类别条件及文本条件的形状生成。核心思想是将形状编码为多尺度小波令牌图,并利用Transformer逐级预测‘更高分辨率的令牌图’。通过将3D AR任务重新定义为‘下一级别’预测,显著降低生成计算成本,同时以更结构化的方式保留关键几何细节。我们在多个主流基准上进行了定量与定性评估,结果表明3D-WAG在覆盖率(Coverage)和最大均值差异(MMD)等关键指标上均表现更优,能生成高保真3D形状,更贴近真实数据分布。

原文摘要 · Abstract (English)

Autoregressive (AR) models have achieved remarkable success in natural language and image generation, but their application to 3D shape modeling remains largely unexplored. Unlike diffusion models, AR models enable more efficient and controllable generation with faster inference times, making them especially suitable for data-intensive domains. Traditional 3D generative models using AR approaches often rely on ``next-token" predictions at the voxel or point level. While effective for certain applications, these methods can be restrictive and computationally expensive when dealing with large-scale 3D data. To tackle these challenges, we introduce 3D-WAG, an AR model for 3D implicit distance fields that can perform unconditional shape generation, class-conditioned and also text-conditioned shape generation. Our key idea is to encode shapes as multi-scale wavelet token maps and use a Transformer to predict the ``next higher-resolution token map" in an autoregressive manner. By redefining 3D AR generation task as ``next-scale" prediction, we reduce the computational cost of generation compared to traditional ``next-token" prediction models, while preserving essential geometric details of 3D shapes in a more structured and hierarchical manner. We evaluate 3D-WAG to showcase its benefit by quantitative and qualitative comparisons with state-of-the-art methods on widely used benchmarks. Our results show 3D-WAG achieves superior performance in key metrics like Coverage and MMD, generating high-fidelity 3D shapes that closely match the real data distribution.

3D生成自回归小波分析隐式建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。