统一评估3D生成与重建的多种表示方法,找最优方案。
Unifi3D: A Study on 3D Representations for Generation and Reconstruction in a Common Framework
- 构建统一框架,对比体素、神经辐射场等3D表示性能。
- 发现重建误差严重影响生成效果,需联合评估。
- 适合研究3D生成、模型选型及系统优化的开发者。
随着文本和图像生成技术的快速发展,研究重点逐渐转向3D生成。与图像中成熟的像素表示不同,3D表示仍多样且碎片化,包括体素网格、神经辐射场、有符号距离函数、点云和八叉树等多种方式,各自具有独特优势与局限性。本文提出一个统一的评估框架,用于衡量不同3D表示在重建与生成任务中的表现。我们从质量、计算效率和泛化能力等多个维度进行对比,并深入实验整个3D生成流程中的关键步骤:预处理、网格重建、自编码器压缩及生成。结果表明,重建误差对整体性能影响显著,强调必须联合评估生成与重建。研究为不同应用场景选择合适的3D模型提供了实用指导,推动更鲁棒、针对性更强的3D生成解决方案发展。代码已开源:https://github.com/isl-org/unifi3d。
原文摘要 · Abstract (English)
Following rapid advancements in text and image generation, research has increasingly shifted towards 3D generation. Unlike the well-established pixel-based representation in images, 3D representations remain diverse and fragmented, encompassing a wide variety of approaches such as voxel grids, neural radiance fields, signed distance functions, point clouds, or octrees, each offering distinct advantages and limitations. In this work, we present a unified evaluation framework designed to assess the performance of 3D representations in reconstruction and generation. We compare these representations based on multiple criteria: quality, computational efficiency, and generalization performance. Beyond standard model benchmarking, our experiments aim to derive best practices over all steps involved in the 3D generation pipeline, including preprocessing, mesh reconstruction, compression with autoencoders, and generation. Our findings highlight that reconstruction errors significantly impact overall performance, underscoring the need to evaluate generation and reconstruction jointly. We provide insights that can inform the selection of suitable 3D models for various applications, facilitating the development of more robust and application-specific solutions in 3D generation. The code for our framework is available at https://github.com/isl-org/unifi3d.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。