解决滚动快门导致的光场图像深度估计失真问题
Dense Scene Reconstruction from Light-Field Images Affected by Rolling Shutter
- 用2D高斯点云分两阶段重建无滚动快门影响的三维形状
- 在真实场景和合成数据上验证，深度图精度显著提升
- 适合做光场三维重建的科研人员和工业应用开发者
本文提出一种从受强滚动快门效应影响的光场（LF）图像中进行密集深度估计的方法。该方法能估计出补偿滚动快门后的视图及稠密的补偿后视差图。基于2D高斯点云渲染与比较策略，提出两阶段框架：第一阶段利用子孔径图像子集估计一个与目标场景形状相关但“受运动影响”的3D形状；第二阶段通过估计可接受的相机运动来计算3D形状的形变。我们在多种场景和运动类型下进行了实验，验证了该方法的有效性与优势。由于缺乏合适的评估数据集，我们还构建了一个精心设计的合成滚动快门光场图像数据集。源代码、训练模型和数据集将公开于：https://github.com/ICB-Vision-AI/DenseRSLF
原文摘要 · Abstract (English)
This paper presents a dense depth estimation approach from light-field (LF) images that is able to compensate for strong rolling shutter (RS) effects. Our method estimates RS compensated views and dense RS compensated disparity maps. We present a two-stage method based on a 2D Gaussians Splatting that allows for a ``render and compare" strategy with a point cloud formulation. In the first stage, a subset of sub-aperture images is used to estimate an RS agnostic 3D shape that is related to the scene target shape ``up to a motion". In the second stage, the deformation of the 3D shape is computed by estimating an admissible camera motion. We demonstrate the effectiveness and advantages of this approach through several experiments conducted for different scenes and types of motions. Due to lack of suitable datasets for evaluation, we also present a new carefully designed synthetic dataset of RS LF images. The source code, trained models and dataset will be made publicly available at: https://github.com/ICB-Vision-AI/DenseRSLF
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。