构建开放平台,用真人投票评估3D生成质量。
3D Arena: An Open Platform for Generative 3D Evaluation
- 通过成对比较收集真人偏好,覆盖19个主流模型
- 真人投票达12.3万次,检测出99.75%用户真实身份
- 发现点云比网格优16.6分,贴图模型比无贴图优144.1分
由于自动化指标与人类对质量的感知不一致,生成式3D模型的评估仍具挑战。现有基准依赖图像指标忽略3D结构,或使用几何度量无法捕捉视觉吸引力和实际用途。为此,我们提出3D Arena,一个基于大规模真人偏好收集的开放式评估平台,采用成对比较方式评估图像到3D生成模型。自2024年6月上线以来,平台已收集8,096名用户提交的123,243张投票,涵盖19个前沿生成3D模型,成为目前最大的真人偏好评估数据集。我们贡献了包含100个评估提示的iso3d数据集,并通过统计欺诈检测实现99.75%的用户真实性验证。基于ELO的排名系统提供可靠模型评估,平台已成为领域公认的评估资源。通过对偏好数据的分析,我们揭示了人类偏好的规律:更倾向视觉呈现特征,其中高斯点云输出相较网格模型获得16.6分优势,带纹理模型相比无纹理模型获得144.1分优势。我们建议改进评估方法,包括多维度评估、任务导向测试及格式感知对比。平台的社区参与使3D Arena成为该领域的基准,同时推动了以人类为中心的生成3D评估研究。
原文摘要 · Abstract (English)
Evaluating Generative 3D models remains challenging due to misalignment between automated metrics and human perception of quality. Current benchmarks rely on image-based metrics that ignore 3D structure or geometric measures that fail to capture perceptual appeal and real-world utility. To address this gap, we present 3D Arena, an open platform for evaluating image-to-3D generation models through large-scale human preference collection using pairwise comparisons. Since launching in June 2024, the platform has collected 123,243 votes from 8,096 users across 19 state-of-the-art models, establishing the largest human preference evaluation for Generative 3D. We contribute the iso3d dataset of 100 evaluation prompts and demonstrate quality control achieving 99.75% user authenticity through statistical fraud detection. Our ELO-based ranking system provides reliable model assessment, with the platform becoming an established evaluation resource. Through analysis of this preference data, we present insights into human preference patterns. Our findings reveal preferences for visual presentation features, with Gaussian splat outputs achieving a 16.6 ELO advantage over meshes and textured models receiving a 144.1 ELO advantage over untextured models. We provide recommendations for improving evaluation methods, including multi-criteria assessment, task-oriented evaluation, and format-aware comparison. The platform's community engagement establishes 3D Arena as a benchmark for the field while advancing understanding of human-centered evaluation in Generative 3D.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。