基于3D高斯的实时人像新视角生成,支持多相机设置和高清细节。
EVA-Gaussian: 3D Gaussian-based Real-time Human Novel View Synthesis under Diverse Multi-view Camera Settings
- 设计跨视角注意力模块,稀疏视角下高效融合多视图信息。
- 在THuman2.0与THumansit数据集上实现高质量渲染,支持多样相机角度。
- 适合需要实时高保真人像重建的虚拟拍摄、VR等应用开发者。
基于前馈的3D高斯点云方法在人像实时新视角合成中表现卓越,但现有方法受限于密集视角配置或图像分辨率上限,难以应对广泛视角差异下的自由视角渲染,且无法在普通GPU上实时恢复精细人体细节。为此,我们提出EVA-Gaussian新框架,支持多样多视角相机设置下的3D人像新视角合成。首先设计高效的跨视角注意力(EVA)模块,在高分辨率输入与稀疏视角条件下实现多视图信息的有效融合,同时最小化时序与计算开销;其次引入特征精炼机制,预测3D高斯属性并为其分配特征值,修正位置估计误差导致的伪影,提升整体视觉保真度。在THuman2.0与THumansit数据集上的实验表明,EVA-Gaussian在多种相机设置下均展现出更优的渲染质量。
原文摘要 · Abstract (English)
Feed-forward based 3D Gaussian Splatting methods have demonstrated exceptional capability in real-time novel view synthesis for human models. However, current approaches are confined to either dense viewpoint configurations or restricted image resolutions. These limitations hinder their flexibility in free-viewpoint rendering across a wide range of camera view angle discrepancies, and also restrict their ability to recover fine-grained human details in real time using commonly available GPUs. To address these challenges, we propose a novel pipeline named EVA-Gaussian for 3D human novel view synthesis across diverse multi-view camera settings. Specifically, we first design an Efficient Cross-View Attention (EVA) module to effectively fuse cross-view information under high resolution inputs and sparse view settings, while minimizing temporal and computational overhead. Additionally, we introduce a feature refinement mechianism to predict the attributes of the 3D Gaussians and assign a feature value to each Gaussian, enabling the correction of artifacts caused by geometric inaccuracies in position estimation and enhancing overall visual fidelity. Experimental results on the THuman2.0 and THumansit datasets showcase the superiority of EVA-Gaussian in rendering quality across diverse camera settings. Project page: https://zhenliuzju.github.io/huyingdong/EVA-Gaussian.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。