将3D生成采样步数从25步缩减至1-2步,速度提升6.5倍以上。
Few-step Flow for 3D Generation via Marginal-Data Transport Distillation
- 通过速度匹配与分布蒸馏,实现流模型的少步高效蒸馏
- 在A800上实现0.68秒延迟,速度提升9.0倍
- 适合追求高效率3D生成的研究者与应用开发者
基于流的3D生成模型通常需要数十次采样步骤。尽管一致性模型(CMs)等少步蒸馏方法在2D扩散模型中取得显著进展,但在更复杂的3D生成任务中仍研究不足。本文提出新型框架MDT-dist,用于少步3D流蒸馏。核心思想是蒸馏预训练模型以学习边缘数据传输(Marginal-Data Transport)。由于速度场积分难以直接实现,我们设计了可优化的目标:速度匹配(VM)和速度蒸馏(VD),分别将优化目标从传输层转换到速度层与分布层。VM稳定匹配师生模型的速度场,但存在偏差梯度;VD进一步利用学习到的速度场进行概率密度蒸馏。在先锋3D生成框架TRELLIS上评估,将每个流变换器的采样步数从25步降至1或2步,在A800上实现0.68秒(1步×2)与0.94秒(2步×2)延迟,分别带来9.0倍和6.5倍加速,同时保持高视觉与几何保真度。大量实验表明,该方法显著优于现有CM蒸馏方法,使TRELLIS在少步3D生成中达到先进性能。
原文摘要 · Abstract (English)
Flow-based 3D generation models typically require dozens of sampling steps during inference. Though few-step distillation methods, particularly Consistency Models (CMs), have achieved substantial advancements in accelerating 2D diffusion models, they remain under-explored for more complex 3D generation tasks. In this study, we propose a novel framework, MDT-dist, for few-step 3D flow distillation. Our approach is built upon a primary objective: distilling the pretrained model to learn the Marginal-Data Transport. Directly learning this objective needs to integrate the velocity fields, while this integral is intractable to be implemented. Therefore, we propose two optimizable objectives, Velocity Matching (VM) and Velocity Distillation (VD), to equivalently convert the optimization target from the transport level to the velocity and the distribution level respectively. Velocity Matching (VM) learns to stably match the velocity fields between the student and the teacher, but inevitably provides biased gradient estimates. Velocity Distillation (VD) further enhances the optimization process by leveraging the learned velocity fields to perform probability density distillation. When evaluated on the pioneer 3D generation framework TRELLIS, our method reduces sampling steps of each flow transformer from 25 to 1 or 2, achieving 0.68s (1 step x 2) and 0.94s (2 steps x 2) latency with 9.0x and 6.5x speedup on A800, while preserving high visual and geometric fidelity. Extensive experiments demonstrate that our method significantly outperforms existing CM distillation methods, and enables TRELLIS to achieve superior performance in few-step 3D generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。