手绘草图一键生成高质量3D模型，精度高且细节丰富。
S3D: Sketch-Driven 3D Model Generation
- 用U-Net将草图转为面分割图，再生成可多视角渲染的3D结构
- 引入风格对齐损失，提升草图与3D输出的一致性，重建更精准
- 适用于设计师快速原型制作，适合低门槛3D建模需求
从2D草图生成高质量3D模型极具挑战，因草图本身存在固有的模糊性和稀疏性。本文提出S3D框架，将手绘草图转换为详细3D模型。该方法采用基于U-Net的编码器-解码器结构，将草图映射为面分割掩码，进而生成可从新视角渲染的3D表示。为确保草图域与3D输出间的一致性，我们设计一种新颖的风格对齐损失，将U-Net瓶颈特征与3D生成模块初始编码器输出对齐，显著提升重建保真度。此外，通过草图数据集增强技术进一步提高网络鲁棒性。该简化框架在草图到3D建模任务中展现出优异性能。项目源代码已公开于https://github.com/hailsong/S3D。
原文摘要 · Abstract (English)
Generating high-quality 3D models from 2D sketches is a challenging task due to the inherent ambiguity and sparsity of sketch data. In this paper, we present S3D, a novel framework that converts simple hand-drawn sketches into detailed 3D models. Our method utilizes a U-Net-based encoder-decoder architecture to convert sketches into face segmentation masks, which are then used to generate a 3D representation that can be rendered from novel views. To ensure robust consistency between the sketch domain and the 3D output, we introduce a novel style-alignment loss that aligns the U-Net bottleneck features with the initial encoder outputs of the 3D generation module, significantly enhancing reconstruction fidelity. To further enhance the network's robustness, we apply augmentation techniques to the sketch dataset. This streamlined framework demonstrates the effectiveness of S3D in generating high-quality 3D models from sketch inputs. The source code for this project is publicly available at https://github.com/hailsong/S3D.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。