arXiv:2412.17808cs.CV2024-12CVPR被引 98

针对3D形状生成中细节丢失问题,提出更高效的小规模高保真编码方法。

Dora: Sampling and Benchmarking for 3D Shape Variational Auto-Encoders

  • 基于边缘复杂度动态采样,聚焦关键几何区域训练
  • 在1280维潜空间实现与超万维编码相当的重建质量
  • 新基准评测突出细粒度几何还原能力,适合生成任务研究者

当前3D内容生成普遍采用变分自编码器(VAE)将形状编码为紧凑潜空间表示以支持扩散生成。然而,现有训练中常用的均匀点采样策略常导致几何细节大量丢失,限制了重建质量和下游生成效果。本文提出Dora-VAE,通过新颖的锐边采样策略与双交叉注意力机制,识别并优先处理高几何复杂度区域,显著提升细微特征保留能力。该策略使模型能聚焦传统均匀采样忽略的关键结构。为系统评估重建质量,我们进一步提出Dora-bench基准,基于锐边密度量化形状复杂度,引入关注显著几何特征重建精度的新指标。在Dora-bench上的大量实验表明,Dora-VAE在仅需1,280维潜空间的情况下,重建质量可媲美最先进的密集型XCube-VAE(>10,000代码),性能提升至少8倍。

原文摘要 · Abstract (English)

Recent 3D content generation pipelines commonly employ Variational Autoencoders (VAEs) to encode shapes into compact latent representations for diffusion-based generation. However, the widely adopted uniform point sampling strategy in Shape VAE training often leads to a significant loss of geometric details, limiting the quality of shape reconstruction and downstream generation tasks. We present Dora-VAE, a novel approach that enhances VAE reconstruction through our proposed sharp edge sampling strategy and a dual cross-attention mechanism. By identifying and prioritizing regions with high geometric complexity during training, our method significantly improves the preservation of fine-grained shape features. Such sampling strategy and the dual attention mechanism enable the VAE to focus on crucial geometric details that are typically missed by uniform sampling approaches. To systematically evaluate VAE reconstruction quality, we additionally propose Dora-bench, a benchmark that quantifies shape complexity through the density of sharp edges, introducing a new metric focused on reconstruction accuracy at these salient geometric features. Extensive experiments on the Dora-bench demonstrate that Dora-VAE achieves comparable reconstruction quality to the state-of-the-art dense XCube-VAE while requiring a latent space at least 8$\times$ smaller (1,280 vs. > 10,000 codes).

3D生成变分自编码器几何细节高效编码

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。