arXiv:2503.23022cs.CV2025-03被引 13

用扩散模型生成可控面数的3D网格,速度比旧方法快35倍。

MeshCraft: Exploring Efficient and Controllable Mesh Generation with Flow-based DiTs

  • 用流模型一次性生成完整网格拓扑,取代逐个生成
  • 800面网格生成仅需3.2秒,速度提升35倍
  • 可指定面数,适合需要精确控制的3D创作

在3D内容生成领域,通过AI模型获得理想网格拓扑一直是3D艺术家的追求。以往方法如MeshGPT采用网格自回归生成,虽视觉效果出色,但因逐令牌预测导致生成极慢且面数不可控。本文提出MeshCraft,一种高效可控的网格生成框架,利用连续空间扩散模型生成离散三角面。其核心由两部分构成:1)基于Transformer的VAE,将原始网格编码为连续面级令牌并重建;2)基于流的扩散Transformer,根据指定面数生成高质量3D网格。通过扩散模型并行生成整个网格拓扑,相比自回归方法显著提升速度与质量。实验表明,生成800面网格仅需3.2秒(较基线快35倍),在ShapeNet和Objaverse数据集上均优于现有方法。同时兼容条件引导策略,可有效减轻艺术家手动建模负担。

原文摘要 · Abstract (English)

In the domain of 3D content creation, achieving optimal mesh topology through AI models has long been a pursuit for 3D artists. Previous methods, such as MeshGPT, have explored the generation of ready-to-use 3D objects via mesh auto-regressive techniques. While these methods produce visually impressive results, their reliance on token-by-token predictions in the auto-regressive process leads to several significant limitations. These include extremely slow generation speeds and an uncontrollable number of mesh faces. In this paper, we introduce MeshCraft, a novel framework for efficient and controllable mesh generation, which leverages continuous spatial diffusion to generate discrete triangle faces. Specifically, MeshCraft consists of two core components: 1) a transformer-based VAE that encodes raw meshes into continuous face-level tokens and decodes them back to the original meshes, and 2) a flow-based diffusion transformer conditioned on the number of faces, enabling the generation of high-quality 3D meshes with a predefined number of faces. By utilizing the diffusion model for the simultaneous generation of the entire mesh topology, MeshCraft achieves high-fidelity mesh generation at significantly faster speeds compared to auto-regressive methods. Specifically, MeshCraft can generate an 800-face mesh in just 3.2 seconds (35$\times$ faster than existing baselines). Extensive experiments demonstrate that MeshCraft outperforms state-of-the-art techniques in both qualitative and quantitative evaluations on ShapeNet dataset and demonstrates superior performance on Objaverse dataset. Moreover, it integrates seamlessly with existing conditional guidance strategies, showcasing its potential to relieve artists from the time-consuming manual work involved in mesh creation.

3D生成扩散模型网格控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。