arXiv:2504.17670cs.CV2025-04被引 8

分离几何与纹理,用法线图提升稀疏视图网格重建精度

DiMeR: Disentangled Mesh Reconstruction Model

  • 将几何与纹理解耦,用法线图预测几何,从RGB图像估计纹理
  • 在GSO和OmniObject3D上,切比雪夫距离降低超30%
  • 支持单图和文本到3D任务,适用于需要高质量网格的生成场景

我们提出DiMeR,一种基于3D监督的新型前馈式解耦网格重建模型,用于稀疏视图网格重建。现有方法面临两大挑战:(i) 纹理可掩盖几何错误,导致几何-纹理联合优化空间中存在多个模糊目标;(ii) 传统网格提取方法冗余、不稳定,缺乏3D监督。为此,我们重新思考网格重建的归纳偏置。首先,将统一的几何-纹理解空间解耦为独立的几何与纹理空间,利用法线图与几何严格一致且精准捕捉表面变化的特点,仅以法线图为输入预测几何,而纹理由RGB图像估计。其次,通过移除低效模块并重设计带3D监督的正则化损失,简化网格提取流程。值得注意的是,DiMeR仍接受原始RGB图像作为输入,借助基础模型预测法线图。大量实验表明,DiMeR在稀疏视图、单图及文生3D任务中均表现优异,显著优于基线,在GSO和OmniObject3D数据集上,切比雪夫距离降低超过30%。

原文摘要 · Abstract (English)

We propose DiMeR, a novel geometry-texture disentangled feed-forward model with 3D supervision for sparse-view mesh reconstruction. Existing methods confront two persistent obstacles: (i) textures can conceal geometric errors, i.e., visually plausible images can be rendered even with wrong geometry, producing multiple ambiguous optimization objectives in geometry-texture mixed solution space for similar objects; and (ii) prevailing mesh extraction methods are redundant, unstable, and lack 3D supervision. To solve these challenges, we rethink the inductive bias for mesh reconstruction. First, we disentangle the unified geometry-texture solution space, where a single input admits multiple feasible solutions, into geometry and texture spaces individually. Specifically, given that normal maps are strictly consistent with geometry and accurately capture surface variations, the normal maps serve as the sole input for geometry prediction in DiMeR, while the texture is estimated from RGB images. Second, we streamline the algorithm of mesh extraction by eliminating modules with low performance/cost ratios and redesigning regularization losses with 3D supervision. Notably, DiMeR still accepts raw RGB images as input by leveraging foundation models for normal prediction. Extensive experiments demonstrate that DiMeR generalises across sparse-view-, single-image-, and text-to-3D tasks, consistently outperforming baselines. On the GSO and OmniObject3D datasets, DiMeR significantly reduces Chamfer Distance by more than 30%.

网格重建解耦建模3D生成法线图

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。