arXiv:2608.01825cs.CVcs.AI2026-08

让3D物体按材质拆分,一次生成所有部分。

PartMat: Material-Aware 3D Part Decomposition with a Single Global Latent

论文配图:PartMat: Material-Aware 3D Part Decomposition with a Single Global Latent
图 1 · 摘自论文原文
  • 用单一全局隐变量统一表示多部件几何,一次前向传播生成全部零件。
  • 在真实材质边界上分解物体,准确率超越现有方法,推理效率高。
  • 适合需要按材质编辑的3D设计场景,如室内装修建模。

近期的局部3D生成方法关注于生成结构化且可编辑的3D资产,但多数仅依据功能语义进行分解,而非实际应用(如室内设计)所需的可编辑材质边界(如布料、木材、金属)。此外,现有方法通常独立生成各部件,导致计算开销随部件数量线性增长。为此,我们提出PartMat,一种高效的材质感知3D部件分解流程,通过单一全局隐变量表示多部件几何。给定参考图像与整体物体几何,PartMat将物体按材质边界分解。首先,提出PartVAE学习该统一表示,并在单次前向传播中解码所有材质部件,从而实现推理成本与部件数解耦。其次,基于此表示训练扩散模型生成部件,并通过强化学习优化材质分配与重叠抑制。最后,为恢复精细几何细节,引入带部件注意力的稀疏体素流匹配模型进行几何后处理。大量实验表明,PartMat在材质感知分解准确率上显著优于现有基线,几何质量相当,同时保持高效推理。

原文摘要 · Abstract (English)

Part-level 3D generation has recently attracted increasing attention for producing structured and editable 3D assets. However, existing methods typically decompose objects according to functional semantics rather than the editable material boundaries (e.g., fabric, wood, metal) required in practical 3D applications such as interior design. Additionally, current methods often generate parts independently, causing computational costs to scale linearly with the part count. To address these limitations, we present PartMat, an efficient material-aware 3D part decomposition pipeline that represents multi-part geometry with a single global latent. Given a reference image and a single whole-object geometry, PartMat decomposes the object into parts that follow material boundaries. First, we propose PartVAE to learn such a unified representation and decode all material parts in a single forward pass, thereby decoupling inference cost from the number of parts. Second, with this representation, a diffusion model is trained for part generation and refined via reinforcement learning for accurate material assignment and overlap suppression. Finally, to recover fine-grained geometric details, we introduce a sparse-voxel flow-matching model with part attention for geometry post-processing. Extensive experiments demonstrate that PartMat significantly outperforms existing baselines in material-aware decomposition accuracy and achieves comparable geometric quality, while maintaining efficient inference.

3D生成材质分解隐变量模型扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。