arXiv:2411.02336cs.CV2024-11CVPR被引 33

让3D模型一键生成一致且无缝的高精度贴图,无需依赖复杂的UV展开。

MVPaint: Synchronized Multi-View Diffusion for Painting Anything 3D

论文配图:MVPaint: Synchronized Multi-View Diffusion for Painting Anything 3D
图 1 · 摘自论文原文
  • 通过多视角同步生成与空间感知修复,实现3D贴图完整覆盖。
  • 在两个新基准上超越现有方法,跨视角一致性显著提升。
  • 适合游戏、影视等需要高质量3D资产生产的场景。

贴图是3D资产制作流程中的关键步骤,能显著提升视觉表现力与多样性。尽管文本到贴图(T2T)生成技术取得进展,现有方法仍存在局部不连续、多视角不一致及严重依赖UV展开结果等问题。为此,我们提出MVPaint——一种新型生成-优化3D贴图框架,可生成高分辨率、无缝且强调多视角一致性的贴图。MVPaint包含三个核心模块:1)同步多视角生成(SMG),基于3D网格同时生成多视图图像,产生因观测缺失导致的未涂色区域;2)空间感知3D修复(S3I),专门用于有效填补此前未观测区域;3)UV优化(UVR),在UV空间中先进行超分辨率重建,再通过空间感知接缝平滑算法修正由UV展开引起的纹理断裂。此外,我们建立了两个T2T评估基准:基于Objaverse数据集精选高质量3D网格的Objaverse T2T基准,以及涵盖整个GSO数据集的GSO T2T基准。大量实验表明,MVPaint在多个指标上优于现有最先进方法,能生成高保真贴图,极少出现Janus问题,且跨视角一致性大幅增强。

原文摘要 · Abstract (English)

Texturing is a crucial step in the 3D asset production workflow, which enhances the visual appeal and diversity of 3D assets. Despite recent advancements in Text-to-Texture (T2T) generation, existing methods often yield subpar results, primarily due to local discontinuities, inconsistencies across multiple views, and their heavy dependence on UV unwrapping outcomes. To tackle these challenges, we propose a novel generation-refinement 3D texturing framework called MVPaint, which can generate high-resolution, seamless textures while emphasizing multi-view consistency. MVPaint mainly consists of three key modules. 1) Synchronized Multi-view Generation (SMG). Given a 3D mesh model, MVPaint first simultaneously generates multi-view images by employing an SMG model, which leads to coarse texturing results with unpainted parts due to missing observations. 2) Spatial-aware 3D Inpainting (S3I). To ensure complete 3D texturing, we introduce the S3I method, specifically designed to effectively texture previously unobserved areas. 3) UV Refinement (UVR). Furthermore, MVPaint employs a UVR module to improve the texture quality in the UV space, which first performs a UV-space Super-Resolution, followed by a Spatial-aware Seam-Smoothing algorithm for revising spatial texturing discontinuities caused by UV unwrapping. Moreover, we establish two T2T evaluation benchmarks: the Objaverse T2T benchmark and the GSO T2T benchmark, based on selected high-quality 3D meshes from the Objaverse dataset and the entire GSO dataset, respectively. Extensive experimental results demonstrate that MVPaint surpasses existing state-of-the-art methods. Notably, MVPaint could generate high-fidelity textures with minimal Janus issues and highly enhanced cross-view consistency.

3D生成贴图生成多视角扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。