为稀疏神经渲染提供高分辨率数据集与可复现评估基准
AIM 2024 Sparse Neural Rendering Challenge: Dataset and Benchmark
- 构建基于DTU MVS格式的97个合成场景，每场景最多64视角、7光照
- 提供82个训练场景和隐藏真值的在线评测平台，支持3/9视图两种稀疏配置
- 解决以往评估不统一、易过拟合问题，助力方法公平比较
近年来，可微分神经渲染在新视角合成、三维重建等任务中取得显著进展，但通常依赖密集视角覆盖以解耦几何与外观。当仅有少量输入视角时（即稀疏神经渲染），问题变得欠定，现有方法多依赖正则化及学习或手工设计的先验。然而，稀疏渲染领域长期缺乏统一、最新的数据集与评估协议：多数研究使用低分辨率图像，数据划分不一致，且测试真值公开导致过拟合风险。本文提出稀疏渲染（SpaRe）数据集与基准评测体系。数据集包含97个基于高质量合成资产的新场景，沿用DTU MVS设置，每个场景含最多64个相机视角与7种光照配置，渲染分辨率为1600×1200。释放82个训练场景，提供在线评测平台，验证与测试集真值保持隐藏。设定两种稀疏配置（3张与9张输入图像），支持可复现评估，并提供公开排行榜，实时更新当前最优性能。
原文摘要 · Abstract (English)
Recent developments in differentiable and neural rendering have made impressive breakthroughs in a variety of 2D and 3D tasks, e.g. novel view synthesis, 3D reconstruction. Typically, differentiable rendering relies on a dense viewpoint coverage of the scene, such that the geometry can be disambiguated from appearance observations alone. Several challenges arise when only a few input views are available, often referred to as sparse or few-shot neural rendering. As this is an underconstrained problem, most existing approaches introduce the use of regularisation, together with a diversity of learnt and hand-crafted priors. A recurring problem in sparse rendering literature is the lack of an homogeneous, up-to-date, dataset and evaluation protocol. While high-resolution datasets are standard in dense reconstruction literature, sparse rendering methods often evaluate with low-resolution images. Additionally, data splits are inconsistent across different manuscripts, and testing ground-truth images are often publicly available, which may lead to over-fitting. In this work, we propose the Sparse Rendering (SpaRe) dataset and benchmark. We introduce a new dataset that follows the setup of the DTU MVS dataset. The dataset is composed of 97 new scenes based on synthetic, high-quality assets. Each scene has up to 64 camera views and 7 lighting configurations, rendered at 1600x1200 resolution. We release a training split of 82 scenes to foster generalizable approaches, and provide an online evaluation platform for the validation and test sets, whose ground-truth images remain hidden. We propose two different sparse configurations (3 and 9 input images respectively). This provides a powerful and convenient tool for reproducible evaluation, and enable researchers easy access to a public leaderboard with the state-of-the-art performance scores. Available at: https://sparebenchmark.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。