用可微分的几何图像实现高保真薄壳3D生成,解决传统方法边界模糊和计算成本高的问题。
DiffGI: Differentiable Geometry Images for High-Fidelity Thin-Shell 3D Generation

- 用连续截断有符号距离场替代二值图,实现亚像素级边界精度
- 提出可微分的Marching Squares算法,支持梯度反传至2D隐空间
- 在32×32隐空间中压缩复杂3D表面,适合衣物等薄壳结构生成
现有3D生成模型多依赖隐式体素表示,难以处理衣物等薄壳与非流形几何。基于几何图像的方法虽以表面为中心,但依赖离散二值占用图,在降采样时产生阶梯伪影并丢失信息,且表面重建为不可微后处理步骤。为此,我们提出可微分几何图像(DiffGI),一个端到端的3D到2D映射框架,无缝集成表面表示与几何优化。DiffGI将二值图替换为连续2D截断有符号距离函数(TSDF),在固定网格分辨率下以亚像素精度编码边界位置,即使在大幅降采样下也无分辨率相关阶梯伪影。在此连续场基础上,我们引入基于解析线性插值的可微分Marching Squares算法,使3D表面损失的梯度可回传至2D隐空间。利用该可微管道,我们训练了带几何感知法向渲染损失的DiffGI-VAE,将复杂3D表面压缩至超紧凑的32×32隐空间,并在此空间上构建基于Transformer的隐空间扩散模型,采用流匹配目标进行条件3D生成。在衣物与物体数据集上的大量实验表明,本方法相比先前几何图像与体素方法在重建保真度与边界精度上均更优,同时显著降低计算资源消耗。
原文摘要 · Abstract (English)
Existing 3D generative models predominantly rely on implicit volumetric representations, which enforce watertight topology and struggle to represent thin-shell and non-manifold geometries such as garments. Geometry image-based approaches offer a surface-centric alternative, but existing methods rely on discrete binary occupancy maps whose resolution-dependent boundary encoding causes staircase artifacts and information loss upon downsampling, while surface reconstruction remains a non-differentiable post-processing step disconnected from the learning pipeline. To address this, we propose Differentiable Geometry Image (DiffGI), an end-to-end 3D-to-2D mapping framework that seamlessly integrates surface representation and geometric optimization. DiffGI replaces binary maps with a continuous 2D Truncated Signed Distance Function (TSDF), which encodes boundary position at subpixel precision within a fixed grid resolution, eliminating resolution-dependent staircase artifacts even under aggressive downsampling. Building on this continuous field, we introduce a differentiable Marching Squares algorithm based on analytical linear interpolation, allowing gradients from 3D surface losses to propagate back to the 2D latent space. Leveraging this differentiable pipeline, we train a DiffGI-VAE augmented with a geometry-aware normal rendering loss to compress complex 3D surfaces into an ultra-compact 32X32 latent space, and instantiate a transformer-based latent diffusion model with a flow-matching objective on top of this space for conditional 3D generation. Extensive experiments on garment and object datasets demonstrate that our method achieves superior reconstruction fidelity and boundary precision compared to prior geometry-image and voxel-based approaches, while requiring significantly fewer computational resources.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。