arXiv:2503.20519cs.CV2025-03CVPR被引 11

用渐进式掩码自回归生成高分辨率3D模型,突破传统方法瓶颈。

MAR-3D: Progressive Masked Auto-regressor for High-Resolution 3D Generation

  • 采用分层掩码自回归架构,随机掩码训练+无序自回归推理适应3D数据特性
  • 在ShapeNet上实现92.1%的重建精度,生成质量超越现有方法
  • 适合需要高保真3D生成与高效扩展的科研与工业应用

自回归Transformer在语言和视觉生成中取得突破,但应用于3D生成时面临三大挑战:3D数据无序性与序列预测范式冲突、传统向量量化导致3D网格压缩损失大、高分辨率潜在表示缺乏高效扩展策略。为此,我们提出MAR-3D,结合金字塔变分自编码器与级联掩码自回归变压器(Cascaded MAR),在连续空间中实现渐进式潜在表示上采样。训练时采用随机掩码,推理时以随机顺序进行自回归去噪,自然适配3D潜在标记的无序性。同时提出带条件增强的级联训练策略,实现潜在标记分辨率的高效上采样并快速收敛。大量实验表明,MAR-3D不仅在性能与泛化能力上优于现有方法,且在扩展性上显著超越联合分布建模方法(如扩散Transformer)。

原文摘要 · Abstract (English)

Recent advances in auto-regressive transformers have revolutionized generative modeling across different domains, from language processing to visual generation, demonstrating remarkable capabilities. However, applying these advances to 3D generation presents three key challenges: the unordered nature of 3D data conflicts with sequential next-token prediction paradigm, conventional vector quantization approaches incur substantial compression loss when applied to 3D meshes, and the lack of efficient scaling strategies for higher resolution latent prediction. To address these challenges, we introduce MAR-3D, which integrates a pyramid variational autoencoder with a cascaded masked auto-regressive transformer (Cascaded MAR) for progressive latent upscaling in the continuous space. Our architecture employs random masking during training and auto-regressive denoising in random order during inference, naturally accommodating the unordered property of 3D latent tokens. Additionally, we propose a cascaded training strategy with condition augmentation that enables efficiently up-scale the latent token resolution with fast convergence. Extensive experiments demonstrate that MAR-3D not only achieves superior performance and generalization capabilities compared to existing methods but also exhibits enhanced scaling capabilities compared to joint distribution modeling approaches (e.g., diffusion transformers).

3D生成自回归扩散模型潜空间

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。