arXiv:2607.24436cs.CV2026-07

用分层点扰动提升3D生成质量,兼顾精度与效率。

MSVS-VAE: Multi-Scale Anchored VecSet for High-Fidelity 3D Reconstruction

论文配图:MSVS-VAE: Multi-Scale Anchored VecSet for High-Fidelity 3D Reconstruction
图 1 · 摘自论文原文
  • 通过分层点扰动逐步加密锚定向量集,增强细节建模能力。
  • 在多个数据集上超越现有方法,解码速度提升10倍,压缩率高10倍。
  • 适合需要高保真、低资源的3D生成任务,如虚拟现实建模。

高保真3D生成模型日益依赖潜在扩散范式,其中3D VAE的重建质量成为主要瓶颈。现有方法主要分为两类:稀疏体素表示虽重建质量高,但内存和计算开销大;集合表示紧凑连续,但因潜在空间稀疏和过度全局平滑,保真度通常较低。我们提出MSVS-VAE,一种分层集合式VAE,有效弥合了保真度差距且不牺牲紧凑性。核心思想是通过分层点扰动逐步加密锚定向量集(anchored VecSet),提升空间容量以支持细粒度几何建模。为高效解码密集化层次,我们用AVS-Conv替代全局交叉注意力,该操作仅在局部邻域内进行几何感知聚合,而非遍历整个潜在集。此外引入多尺度查询解码,融合粗到细的潜在特征:粗尺度提供稳定全局上下文,细尺度细化局部几何,减少过小感受野带来的伪影。在Objaverse、ABO及真实场景基准上的大量实验表明,MSVS-VAE持续优于先前集合与体素基方法,解码速度较以往集合方法快约10倍,紧凑性较体素基基线高出约10倍。

原文摘要 · Abstract (English)

High-fidelity 3D generative modeling increasingly relies on the latent diffusion paradigm, where the reconstruction quality of the underlying 3D VAE becomes a primary bottleneck. Existing approaches largely follow two paradigms: sparse voxel-based representations achieve strong reconstruction quality but incur significant memory and computational overhead, while set-based representations are compact and continuous yet typically lag in fidelity due to latent sparsity and excessive global smoothness. We propose MSVS-VAE, a hierarchical set-based VAE that closes this fidelity gap without sacrificing compactness. Our key idea is to progressively densify anchored VecSet latents via hierarchical point-shuffle upsampling, increasing spatial capacity for fine-grained geometry modeling. To efficiently decode from the densified hierarchy, we replace global cross-attention with AVS-Conv, a geometry-aware local aggregation operator operating within local neighborhoods rather than the exhaustive latent set. We further introduce multi-scale query decoding to fuse coarse-to-fine latent features, where coarse scales provide stable global context, and fine scales refine localized geometry, reducing artifacts from overly local receptive fields. Extensive experiments on Objaverse, ABO, and in-the-wild benchmarks demonstrate that MSVS-VAE consistently outperforms prior set-based and voxel-based VAEs, delivering approximately 10x faster decoding than prior set-based methods and approximately 10x higher compactness than voxel-based baselines.

3D生成生成模型高效架构

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。