arXiv:2503.16302cs.CVcs.AI2025-03ICCV被引 27

让3D形状生成快45倍,同时保持高质量。

Unleashing Vecset Diffusion Model for Fast Shape Generation

  • 用渐进流蒸馏稳定扩散采样,5步即可完成生成。
  • 新解码器降低计算量,重建速度提升45倍以上。
  • 适合需要快速生成3D模型的工业与设计场景。

3D形状生成得益于原生3D扩散模型的发展,尤其是向量集扩散模型(VDM)。尽管近期成果在高分辨率生成上表现良好,但VDM仍面临高速生成难题,主要源于扩散采样和VAE解码两方面的加速困难,此前研究未充分探索。为此,本文提出FlashVDM,系统性加速VDM中的VAE与DiT模块。对于DiT,FlashVDM通过引入渐进流蒸馏实现稳定一致性蒸馏,支持仅5步推理且质量相当;对于VAE,提出闪电向量集解码器,包含自适应键值选择、分层体素解码与高效网络设计,利用向量集局部性及表面稀疏性显著降低浮点运算量,大幅减少解码开销。将FlashVDM应用于Hunyuan3D-2,得到Hunyuan3D-2 Turbo。系统评估表明,该模型显著优于现有快速3D生成方法,在重建任务上推理时间缩短超45倍,在生成任务上缩短32倍,性能媲美最先进水平。代码与模型已开源:https://github.com/Tencent/FlashVDM。

原文摘要 · Abstract (English)

3D shape generation has greatly flourished through the development of so-called "native" 3D diffusion, particularly through the Vecset Diffusion Model (VDM). While recent advancements have shown promising results in generating high-resolution 3D shapes, VDM still struggles with high-speed generation. Challenges exist because of difficulties not only in accelerating diffusion sampling but also VAE decoding in VDM, areas under-explored in previous works. To address these challenges, we present FlashVDM, a systematic framework for accelerating both VAE and DiT in VDM. For DiT, FlashVDM enables flexible diffusion sampling with as few as 5 inference steps and comparable quality, which is made possible by stabilizing consistency distillation with our newly introduced Progressive Flow Distillation. For VAE, we introduce a lightning vecset decoder equipped with Adaptive KV Selection, Hierarchical Volume Decoding, and Efficient Network Design. By exploiting the locality of the vecset and the sparsity of shape surface in the volume, our decoder drastically lowers FLOPs, minimizing the overall decoding overhead. We apply FlashVDM to Hunyuan3D-2 to obtain Hunyuan3D-2 Turbo. Through systematic evaluation, we show that our model significantly outperforms existing fast 3D generation methods, achieving comparable performance to the state-of-the-art while reducing inference time by over 45x for reconstruction and 32x for generation. Code and models are available at https://github.com/Tencent/FlashVDM.

3D生成扩散模型加速推理向量集

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。