用户可局部涂抹风格,让粗糙3D模型自动生成细节丰富的高精度几何体。
DECOLLAGE: 3D Detailization by Controllable, Localized, and Learned Geometry Enhancement
- 通过掩码感知的金字塔GAN实现风格的可控局部增强
- 在不同区域保持结构完整性的同时融合多样风格,生成连贯细节
- 适合交互式3D创作,支持从零开始建模的创意工作流
我们提出一种3D建模方法,使用户可通过机器学习对3D形状进行精细化处理,拓展AI辅助3D内容创作的能力。给定一个粗略体素形状(如简单拉伸工具生成或生成模型输出),用户可直接在形状不同区域“涂抹”来自示例形状的目标风格,以表达引人注目的几何细节。这些区域随后被上采样为高分辨率几何体,且严格遵循所涂风格。为实现这种可控、局部化的3D细节化,我们在金字塔生成对抗网络(Pyramid GAN)基础上引入掩码感知机制,并设计新型结构损失与先验,确保即使所借用风格来自不同语义部件或不同类别,仍能同时保留期望的粗粒度结构和精细特征。大量实验表明,该方法支持新颖的交互式创作流程与应用;相比基于全局细节化的现有技术,本方法生成的高分辨率风格化几何体具有更强的结构保持性、更连贯的细节表现和更自然的风格过渡。
原文摘要 · Abstract (English)
We present a 3D modeling method which enables end-users to refine or detailize 3D shapes using machine learning, expanding the capabilities of AI-assisted 3D content creation. Given a coarse voxel shape (e.g., one produced with a simple box extrusion tool or via generative modeling), a user can directly "paint" desired target styles representing compelling geometric details, from input exemplar shapes, over different regions of the coarse shape. These regions are then up-sampled into high-resolution geometries which adhere with the painted styles. To achieve such controllable and localized 3D detailization, we build on top of a Pyramid GAN by making it masking-aware. We devise novel structural losses and priors to ensure that our method preserves both desired coarse structures and fine-grained features even if the painted styles are borrowed from diverse sources, e.g., different semantic parts and even different shape categories. Through extensive experiments, we show that our ability to localize details enables novel interactive creative workflows and applications. Our experiments further demonstrate that in comparison to prior techniques built on global detailization, our method generates structure-preserving, high-resolution stylized geometries with more coherent shape details and style transitions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。