用类别提示生成结构一致的3D新物体，解决纹理与形状不匹配问题。
Category-Aware 3D Object Composition with Disentangled Texture and Shape Multi-view Diffusion
- 输入3D模型+另一类物体文本，通过多视角扩散生成新2D图像。
- 引入纹理与形状双扩散模块，保持多视角纹理一致性和形状准确性。
- 适合需要高质量3D组合生成的研究者或设计师使用。
本文提出一种全新的3D物体合成任务：将一个3D模型与另一类物体组合生成新颖的3D模型。现有文本/图像/3D-to-3D方法难以有效融合多源内容，常导致纹理不一致和形状失真。为此，我们提出简单而强大的类别+3D-to-3D（C33D）方法，先从输入3D模型渲染多视角图像与法线图，再以前景图与另一类物体的文本描述为输入，通过自适应文本-图像和谐机制（ATIH）生成新颖2D物体。为确保纹理一致性，引入纹理多视角扩散，基于新生成的2D物体优化其余多视角RGB图像的纹理。为提升形状准确性，提出形状多视角扩散，同时优化多视角RGB图像与法线图的二维形状，均以新2D物体为条件。最终重建完整且新颖的3D模型。大量实验表明方法有效，生成效果惊艳，例如图1第一行所示的鲨鱼(3D)-鳄鱼(文本)组合。项目页面见：https://xzr52.github.io/C33D/
原文摘要 · Abstract (English)
In this paper, we tackle a new task of 3D object synthesis, where a 3D model is composited with another object category to create a novel 3D model. However, most existing text/image/3D-to-3D methods struggle to effectively integrate multiple content sources, often resulting in inconsistent textures and inaccurate shapes. To overcome these challenges, we propose a straightforward yet powerful approach, category+3D-to-3D (C33D), for generating novel and structurally coherent 3D models. Our method begins by rendering multi-view images and normal maps from the input 3D model, then generating a novel 2D object using adaptive text-image harmony (ATIH) with the front-view image and a text description from another object category as inputs. To ensure texture consistency, we introduce texture multi-view diffusion, which refines the textures of the remaining multi-view RGB images based on the novel 2D object. For enhanced shape accuracy, we propose shape multi-view diffusion to improve the 2D shapes of both the multi-view RGB images and the normal maps, also conditioned on the novel 2D object. Finally, these outputs are used to reconstruct a complete and novel 3D model. Extensive experiments demonstrate the effectiveness of our method, yielding impressive 3D creations, such as shark(3D)-crocodile(text) in the first row of Fig. 1. A project page is available at: https://xzr52.github.io/C33D/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。