构建首个3D生成模型综合评估基准,实现人机评价对齐。
3DGen-Bench: Comprehensive Benchmark Suite for 3D Generative Models
- 设计对抗式平台3DGen-Arena收集人类偏好数据
- 建立大规模多维度偏好数据集3DGen-Bench,支持文本/图像到3D生成评估
- 开发基于CLIP和多模态大模型的自动评估系统,预测人类评分更准确
3D生成技术快速发展,但评估体系滞后,如何使自动评估与人类感知一致成为关键挑战。现有语言与图像生成领域已探索人类偏好并展现良好拟合能力,但3D领域仍缺乏此类全面的偏好数据集。为此,我们构建了3DGen-Arena对抗式平台,精心设计多样文本与图像提示,通过公众用户与专家标注者收集人类偏好,形成大规模多维度人类偏好数据集3DGen-Bench。基于该数据集,我们训练了基于CLIP的评分模型3DGen-Score和基于多模态大模型的自动评估器3DGen-Eval。二者创新性地统一了文本到3D与图像到3D生成的质量评估,结合各自优势构成自动化评估系统。大量实验表明,该评分模型在预测人类偏好方面表现优异,与人类排序的相关性显著优于现有指标。我们认为3DGen-Bench数据集与自动化评估系统将推动3D生成领域的公平评估,促进生成模型及其下游应用的发展。
原文摘要 · Abstract (English)
3D generation is experiencing rapid advancements, while the development of 3D evaluation has not kept pace. How to keep automatic evaluation equitably aligned with human perception has become a well-recognized challenge. Recent advances in the field of language and image generation have explored human preferences and showcased respectable fitting ability. However, the 3D domain still lacks such a comprehensive preference dataset over generative models. To mitigate this absence, we develop 3DGen-Arena, an integrated platform in a battle manner. Then, we carefully design diverse text and image prompts and leverage the arena platform to gather human preferences from both public users and expert annotators, resulting in a large-scale multi-dimension human preference dataset 3DGen-Bench. Using this dataset, we further train a CLIP-based scoring model, 3DGen-Score, and a MLLM-based automatic evaluator, 3DGen-Eval. These two models innovatively unify the quality evaluation of text-to-3D and image-to-3D generation, and jointly form our automated evaluation system with their respective strengths. Extensive experiments demonstrate the efficacy of our scoring model in predicting human preferences, exhibiting a superior correlation with human ranks compared to existing metrics. We believe that our 3DGen-Bench dataset and automated evaluation system will foster a more equitable evaluation in the field of 3D generation, further promoting the development of 3D generative models and their downstream applications. Project page is available at https://zyh482.github.io/3DGen-Bench/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。