arXiv:2601.12391cs.CV2026-01AAAI

提出新模型,直接生成点云场景,无需外部数据库。

Class-Partitioned VQ-VAE and Latent Flow Matching for Point Cloud Scene Generation

  • 按类别划分码本,提升点云解码准确性。
  • 在复杂客厅场景中,切比雪夫误差降低70.4%。
  • 适合需要高质量点云生成的研究者。

现有3D场景生成方法多仅生成物体边界框参数,而基于扩散的方法虽生成类别标签与潜在特征,仍需从预定义数据库中检索物体。对于包含多种类别的复杂场景,当前自编码器难以有效解码扩散生成的潜在表示为符合目标类别的点云。本文提出类分区向量量化变分自编码器(CPVQ-VAE),采用按类别划分的码本结构,使码向量与类别绑定。为解决码本坍缩问题,设计类感知的动态更新策略,对各类别分区内的失效码向量进行重初始化。推理时,由专为场景生成设计的潜在空间流匹配模型(LFMM)生成对象特征与类别标签,输入至CPVQ-VAE;其类感知反向查找将生成的潜在表示映射至对应类别的码本条目,并解码为特定类别的点云形状。由此实现不依赖外部物体数据库的纯点云生成。大量实验表明,该方法在复杂客厅场景中可分别降低70.4%和72.3%的切比雪夫距离与点到网格误差,可靠重建合理点云场景。

原文摘要 · Abstract (English)

Most 3D scene generation methods are limited to only generating object bounding box parameters while newer diffusion methods also generate class labels and latent features. Using object size or latent feature, they then retrieve objects from a predefined database. For complex scenes of varied, multi-categorical objects, diffusion-based latents cannot be effectively decoded by current autoencoders into the correct point cloud objects which agree with target classes. We introduce a Class-Partitioned Vector Quantized Variational Autoencoder (CPVQ-VAE) that is trained to effectively decode object latent features, by employing a pioneering $\textit{class-partitioned codebook}$ where codevectors are labeled by class. To address the problem of $\textit{codebook collapse}$, we propose a $\textit{class-aware}$ running average update which reinitializes dead codevectors within each partition. During inference, object features and class labels, both generated by a Latent-space Flow Matching Model (LFMM) designed specifically for scene generation, are consumed by the CPVQ-VAE. The CPVQ-VAE's class-aware inverse look-up then maps generated latents to codebook entries that are decoded to class-specific point cloud shapes. Thereby, we achieve pure point cloud generation without relying on an external objects database for retrieval. Extensive experiments reveal that our method reliably recovers plausible point cloud scenes, with up to 70.4% and 72.3% reduction in Chamfer and Point2Mesh errors on complex living room scenes.

点云生成自编码器扩散模型类别分区

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。