对比高斯点云与NeRF在动态场景中的视觉感知质量,揭示两者优劣。
Evaluating Human Perception of Novel View Synthesis: Subjective Quality Assessment of Gaussian Splatting and NeRF in Dynamic Scenes
- 设计双主观实验评估动态场景下两种视图合成技术的视觉质量。
- 首次系统研究移动物体对合成效果感知的影响,覆盖多视角真实视频。
- 发现现有客观指标无法准确反映人眼感受,为新评测标准提供依据。
高斯点云(Gaussian Splatting, GS)和神经辐射场(Neural Radiance Fields, NeRF)是革新视图合成(NVS)领域的两项关键技术,能从稀疏视角图像中生成沉浸式逼真渲染效果。其在虚拟现实、三维建模及医学器官成像等场景中具有重要应用,因此从人类感知角度评估其质量至关重要。尽管已有研究开展过主观评估,但仍面临方法选择、场景覆盖与评估范式不足等问题。为此,本研究针对基于GS与NeRF的NVS方法,在动态真实场景中开展两项主观实验,涵盖360°、前向及单视角视频,覆盖更丰富的真实场景。首次探索了动态物体对视图合成感知质量的影响。两组实验有助于全面理解不同观看路径下的人类感知影响,推动全参考与无参考质量评估指标的发展。同时,我们在所构建数据库上建立了一个全面的客观指标基准,结果表明现有方法仍难以准确捕捉主观质量。研究揭示了现有NVS方法的局限性,为新方法发展提供方向。
原文摘要 · Abstract (English)
Gaussian Splatting (GS) and Neural Radiance Fields (NeRF) are two groundbreaking technologies that have revolutionized the field of Novel View Synthesis (NVS), enabling immersive photorealistic rendering and user experiences by synthesizing multiple viewpoints from a set of images of sparse views. The potential applications of NVS, such as high-quality virtual and augmented reality, detailed 3D modeling, and realistic medical organ imaging, underscore the importance of quality assessment of NVS methods from the perspective of human perception. Although some previous studies have explored subjective quality assessments for NVS technology, they still face several challenges, especially in NVS methods selection, scenario coverage, and evaluation methodology. To address these challenges, we conducted two subjective experiments for the quality assessment of NVS technologies containing both GS-based and NeRF-based methods, focusing on dynamic and real-world scenes. This study covers 360°, front-facing, and single-viewpoint videos while providing a richer and greater number of real scenes. Meanwhile, it's the first time to explore the impact of NVS methods in dynamic scenes with moving objects. The two types of subjective experiments help to fully comprehend the influences of different viewing paths from a human perception perspective and pave the way for future development of full-reference and no-reference quality metrics. In addition, we established a comprehensive benchmark of various state-of-the-art objective metrics on the proposed database, highlighting that existing methods still struggle to accurately capture subjective quality. The results give us some insights into the limitations of existing NVS methods and may promote the development of new NVS methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。