构建3D人脸生成评估基准，用大模型实现真实感与质量精准评分。
LMME3DHF: Benchmarking and Evaluating Multimodal 3D Human Face Generation with LMMs
- 基于大模型设计多模态评估方法，支持评分与失真定位。
- 在2000个视频上实现与人类判断高度一致的评分准确率。
- 适合研究生成质量评估、3D人脸应用的开发者与研究人员。
生成式人工智能的快速发展推动了3D人面部（HF）在媒体制作、虚拟现实、安全、医疗及游戏开发等领域的应用。然而，由于人类感知的主观性以及对人脸特征的敏感性，评估这些AI生成3D人脸的质量与真实感仍面临挑战。为此，我们开展了一项全面的研究，提出Gen3DHF——一个包含2,000个AI生成3D人脸视频、4,000个平均意见得分（MOS），涵盖质量与真实性两个维度，并附带2,000张失真感知显著图和失真描述的大规模基准数据集。基于此，我们提出LMME3DHF，一种基于大型多模态模型（LMM）的3DHF评估指标，可预测质量与真实性分数、进行失真感知的视觉问答及失真感知显著区域预测。实验表明，LMME3DHF在准确预测生成3D人脸质量分数、有效识别失真显著区域与失真类型方面均达到当前最优水平，且与人类感知判断高度一致。该研究的Gen3DHF数据库与LMME3DHF模型将在论文发表后公开。
原文摘要 · Abstract (English)
The rapid advancement in generative artificial intelligence have enabled the creation of 3D human faces (HFs) for applications including media production, virtual reality, security, healthcare, and game development, etc. However, assessing the quality and realism of these AI-generated 3D human faces remains a significant challenge due to the subjective nature of human perception and innate perceptual sensitivity to facial features. To this end, we conduct a comprehensive study on the quality assessment of AI-generated 3D human faces. We first introduce Gen3DHF, a large-scale benchmark comprising 2,000 videos of AI-Generated 3D Human Faces along with 4,000 Mean Opinion Scores (MOS) collected across two dimensions, i.e., quality and authenticity, 2,000 distortion-aware saliency maps and distortion descriptions. Based on Gen3DHF, we propose LMME3DHF, a Large Multimodal Model (LMM)-based metric for Evaluating 3DHF capable of quality and authenticity score prediction, distortion-aware visual question answering, and distortion-aware saliency prediction. Experimental results show that LMME3DHF achieves state-of-the-art performance, surpassing existing methods in both accurately predicting quality scores for AI-generated 3D human faces and effectively identifying distortion-aware salient regions and distortion types, while maintaining strong alignment with human perceptual judgments. Both the Gen3DHF database and the LMME3DHF will be released upon the publication.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。