主动选择稀疏区域视角,用生成修复提升4D高斯渲染质量
FillGS: Filling Observation Gaps in 4D Gaussian Splatting via Viewpoint-Time Selection and Generative Refinement

- 基于渲染敏感度和运动感知密度选虚拟视角
- 修复后图像过滤不可靠区域,仅用可信部分微调模型
- 适合动态大运动场景的高质量重建,尤其缺视角时
4D高斯点阵(4DGS)能实现动态场景的逼真渲染,但视角覆盖有限时,部分时空区域观测稀疏,易产生伪影,尤其在大运动场景中。现有方法依赖启发式虚拟视角选择,再进行生成修复,无法主动探索稀疏区域。为此,本文提出一种主动选择时空虚拟视角的修复流程:根据4D高斯的渲染敏感度与运动感知观测密度,优先选择能缓解观测稀疏性的视角;在生成修复后的图像中,剔除与真实观测冲突或可能含生成伪影的区域,并仅用可靠区域微调4DGS。我们在多视角视频基准上评估该方法,采用新设计的训练/测试划分以制造观测缺口。结果表明,无论定性还是定量评价,本方法均优于先前视角选择策略与微调方法,且显著减少伪影。
原文摘要 · Abstract (English)
4D Gaussian Splatting (4DGS) can render dynamic scenes photorealistically. However, with limited viewpoint coverage, some spatiotemporal regions remain sparsely observed, leading to artifacts, particularly in scenes with large motion. Existing approaches leveraging generative models rely on heuristic virtual-viewpoint selection before refining rendered views. As a result, they cannot actively explore such sparsely observed regions. To address this issue, we propose a pipeline that actively selects spatiotemporal virtual viewpoints to improve 4DGS reconstruction. Our method selects virtual viewpoints for generative enhancement based on the rendering sensitivity and motion-aware observation density of 4D Gaussians, prioritizing views that alleviate observation sparsity. In the refined images, we filter out regions that conflict with captured observations or are likely to contain generative artifacts and then fine-tune 4DGS using only the reliable regions. We evaluate our method on multi-view video benchmarks using new train/test splits designed to induce observation gaps. Results show consistent improvements over prior viewpoint selection strategies and fine-tuning methods in both qualitative and quantitative evaluations, while reducing artifacts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。