让3D模型可局部编辑，改一张图就能调整模型细节。
CMD: Controllable Multiview Diffusion for 3D Editing and Progressive Generation
- 用多视角扩散模型，根据已有部分生成新组件。
- 局部修改输入图像，仅影响对应3D部分，其他不变。
- 适合需要精细调控3D模型的设计师和开发者。
近期3D生成方法已展现出自动化创建3D模型的强大能力。然而，大多数方法仅依赖输入图像或文本提示生成3D模型，缺乏对生成模型各组成部分的可控性。输入图像的任何修改都会导致整个3D模型重新生成。本文提出一种名为CMD的新方法，能够从输入图像生成3D模型，并支持对模型各组件进行灵活的局部编辑。在CMD中，将3D生成建模为条件多视角扩散模型，以已知部分作为条件，生成待编辑或新增的组件。该条件多视角扩散模型不仅支持分部件生成3D模型，还能根据输入图像的局部修改实现3D模型的局部编辑，而无需改变其他部分。大量实验表明，CMD将复杂的3D生成任务分解为多个组件，提升了生成质量；同时，仅通过编辑一张渲染图像即可实现高效、灵活的3D模型局部修改。
原文摘要 · Abstract (English)
Recently, 3D generation methods have shown their powerful ability to automate 3D model creation. However, most 3D generation methods only rely on an input image or a text prompt to generate a 3D model, which lacks the control of each component of the generated 3D model. Any modifications of the input image lead to an entire regeneration of the 3D models. In this paper, we introduce a new method called CMD that generates a 3D model from an input image while enabling flexible local editing of each component of the 3D model. In CMD, we formulate the 3D generation as a conditional multiview diffusion model, which takes the existing or known parts as conditions and generates the edited or added components. This conditional multiview diffusion model not only allows the generation of 3D models part by part but also enables local editing of 3D models according to the local revision of the input image without changing other 3D parts. Extensive experiments are conducted to demonstrate that CMD decomposes a complex 3D generation task into multiple components, improving the generation quality. Meanwhile, CMD enables efficient and flexible local editing of a 3D model by just editing one rendered image.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。