用可微渲染重建复杂透明物体的几何与材质,效果优于现有方法。
DiffTrans: Differentiable Geometry-Materials Decomposition for Reconstructing Transparent Objects
- 基于可微光线追踪,统一优化几何、折射率和吸收率。
- 在多视角图像下实现复杂拓扑透明物体的高精度重建。
- 支持复杂纹理和多样拓扑,适合真实场景应用。
从多视角图像重建透明物体是一项挑战性任务,因其光传播行为复杂且难以确定。传统方法通常仅适用于特定场景,如均匀拓扑、理想透明或仅有表面材质的情况,限制了实际应用。本文提出一种名为DiffTrans的可微渲染框架,能够高效分解并重建透明物体的几何与材质,在包含复杂拓扑和纹理的复杂场景中实现高精度重建。首先,利用带膨胀和平滑正则化的FlexiCubes作为等值面表示,快速从多视角轮廓重建初始几何;同时,采用环境光辐射场恢复场景光照。随后,设计递归可微光线追踪器,以端到端方式联合优化几何、折射率与吸收率,显著提升重建质量。该光线追踪器可基于CUDA实现,大幅降低计算成本。大量实验表明,DiffTrans在多个基准测试中均优于现有方法,尤其在复杂场景下的透明物体重建表现优异。代码已开源:https://github.com/lcp29/DiffTrans。
原文摘要 · Abstract (English)
Reconstructing transparent objects from a set of multi-view images is a challenging task due to the complicated nature and indeterminate behavior of light propagation. Typical methods are primarily tailored to specific scenarios, such as objects following a uniform topology, exhibiting ideal transparency and surface specular reflections, or with only surface materials, which substantially constrains their practical applicability in real-world settings. In this work, we propose a differentiable rendering framework for transparent objects, dubbed DiffTrans, which allows for efficient decomposition and reconstruction of the geometry and materials of transparent objects, thereby reconstructing transparent objects accurately in intricate scenes with diverse topology and complex texture. Specifically, we first utilize FlexiCubes with dilation and smoothness regularization as the iso-surface representation to reconstruct an initial geometry efficiently from the multi-view object silhouette. Meanwhile, we employ the environment light radiance field to recover the environment of the scene. Then we devise a recursive differentiable ray tracer to further optimize the geometry, index of refraction and absorption rate simultaneously in a unified and end-to-end manner, leading to high-quality reconstruction of transparent objects in intricate scenes. A prominent advantage of the designed ray tracer is that it can be implemented in CUDA, enabling a significantly reduced computational cost. Extensive experiments on multiple benchmarks demonstrate the superior reconstruction performance of our DiffTrans compared with other methods, especially in intricate scenes involving transparent objects with diverse topology and complex texture. The code is available at https://github.com/lcp29/DiffTrans.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。