构建首个用于折射反射物体重建的合成数据集与基准测试
RefRef: A Synthetic Dataset and Benchmark for Reconstructing Refractive and Reflective Objects
- 构建包含150个场景的合成数据集，涵盖复杂折射反射物体
- 提出基于折射率的最优光照路径计算方法，显著提升重建精度
- 为视觉重建领域提供新挑战，适合研究材质建模与神经渲染者参考
当前3D重建与新视角生成方法在不透明朗伯物体上表现良好，但大多假设光线直线传播，难以处理折射与反射材质。现有专门针对此类效应的数据集稀缺，制约了性能评估与技术发展。本文提出一个合成的RefRef数据集及基准测试，包含50个不同复杂度的折射反射物体（从单材料凸形到多材料非凸形），每个物体置于三种背景中，共150个场景。我们还设计了一种基于物体几何与折射率的最优光照路径计算方法（oracle方法），并基于此提出无需假设光路的重建方案。在多个前沿方法上的基准测试显示，所有方法均显著落后于oracle，凸显该任务的难度与数据集的有效性。
原文摘要 · Abstract (English)
Modern 3D reconstruction and novel view synthesis approaches have demonstrated strong performance on scenes with opaque Lambertian objects. However, most assume straight light paths and therefore cannot properly handle refractive and reflective materials. Moreover, datasets specialized for these effects are limited, stymieing efforts to evaluate performance and develop suitable techniques. In this work, we introduce a synthetic RefRef dataset and benchmark for reconstructing scenes with refractive and reflective objects from posed images. Our dataset has 50 such objects of varying complexity, from single-material convex shapes to multi-material non-convex shapes, each placed in three different background types, resulting in 150 scenes. We also propose an oracle method that, given the object geometry and refractive indices, calculates accurate light paths for neural rendering, and an approach based on this that avoids these assumptions. We benchmark these against several state-of-the-art methods and show that all methods lag significantly behind the oracle, highlighting the challenges of the task and dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。