构建2000个高保真3D数字孪生数据集,支持真实世界重建评估
Digital Twin Catalog: A Large-Scale Photorealistic 3D Object Digital Twin Dataset
- 采集2000个物体的多视角图像,涵盖不同光照与佩戴式设备拍摄
- 首次提供基于真实场景的3D数字孪生评估基准,支持方法对比与优化
- 适合研究3D重建、数字孪生及增强现实应用的开发者与学者
我们提出Digital Twin Catalog(DTC)数据集,包含2000个经扫描的高质量3D数字孪生物体,以及使用DSLR相机和头戴式AR眼镜在不同光照条件下捕获的图像序列。数字孪生是物理对象的高度精确虚拟映射,准确还原其形状、外观、物理属性等特征。尽管基于神经网络的3D重建与逆向渲染技术已显著提升重建质量,但缺乏大规模、真实世界级别的数字孪生级数据集与基准测试。此外,为推动3D数字孪生普及,需将其与下一代以用户为中心的计算平台(如AR眼镜)结合。目前尚无针对此类设备图像的3D重建评估数据集。DTC填补了这一空白,建立了首个综合性真实世界评估基准,为比较和改进现有重建方法提供了坚实基础。数据集已公开于https://www.projectaria.com/datasets/dtc/,并开放基线评估代码。
原文摘要 · Abstract (English)
We introduce the Digital Twin Catalog (DTC), a new large-scale photorealistic 3D object digital twin dataset. A digital twin of a 3D object is a highly detailed, virtually indistinguishable representation of a physical object, accurately capturing its shape, appearance, physical properties, and other attributes. Recent advances in neural-based 3D reconstruction and inverse rendering have significantly improved the quality of 3D object reconstruction. Despite these advancements, there remains a lack of a large-scale, digital twin-quality real-world dataset and benchmark that can quantitatively assess and compare the performance of different reconstruction methods, as well as improve reconstruction quality through training or fine-tuning. Moreover, to democratize 3D digital twin creation, it is essential to integrate creation techniques with next-generation egocentric computing platforms, such as AR glasses. Currently, there is no dataset available to evaluate 3D object reconstruction using egocentric captured images. To address these gaps, the DTC dataset features 2,000 scanned digital twin-quality 3D objects, along with image sequences captured under different lighting conditions using DSLR cameras and egocentric AR glasses. This dataset establishes the first comprehensive real-world evaluation benchmark for 3D digital twin creation tasks, offering a robust foundation for comparing and improving existing reconstruction methods. The DTC dataset is already released at https://www.projectaria.com/datasets/dtc/ and we will also make the baseline evaluations open-source.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。