从草图生成3D场景，仅需单张草图即可完成高质量重建。
3D Reconstruction from Sketches
- 通过对应点拼接多张草图，再用CycleGAN转为真实图像。
- 基于MegaDepth模型估计深度图，实现从单草图生成3D结构。
- 构建了图像-草图配对数据集，适用于多种手绘风格的重建。
本文研究从多张草图重建3D场景的问题，提出一个三步流程：(1) 利用对应点拼接多张草图；(2) 采用CycleGAN将拼接后的草图转换为逼真图像；(3) 使用预训练的MegaDepth网络估计该图像的深度图以生成3D重建。研究贡献包括构建一个图像-草图配对数据集，其中图像来自苏黎世建筑数据库，草图由作者生成，并用于训练第二步中的CycleGAN。尽管拼接过程在真实手绘草图上泛化能力较弱，但整个管道在单张草图输入下对多种草图风格均表现良好。
原文摘要 · Abstract (English)
We consider the problem of reconstructing a 3D scene from multiple sketches. We propose a pipeline which involves (1) stitching together multiple sketches through use of correspondence points, (2) converting the stitched sketch into a realistic image using a CycleGAN, and (3) estimating that image's depth-map using a pre-trained convolutional neural network based architecture called MegaDepth. Our contribution includes constructing a dataset of image-sketch pairs, the images for which are from the Zurich Building Database, and sketches have been generated by us. We use this dataset to train a CycleGAN for our pipeline's second step. We end up with a stitching process that does not generalize well to real drawings, but the rest of the pipeline that creates a 3D reconstruction from a single sketch performs quite well on a wide variety of drawings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。