arXiv:2506.04908cs.CV2025-06被引 1

用3D高斯点云生成立体数据集,提升模型泛化能力

Generating Synthetic Stereo Datasets using 3D Gaussian Splatting and Expert Knowledge Transfer

  • 基于3D高斯点云与专家知识迁移生成立体图像数据
  • 使用FoundationStereo的视差估计使生成数据更清晰,提升零样本泛化性能
  • 适合需要快速训练深色立体模型的研究者

本文提出一种基于3D高斯点云(3DGS)的立体数据集生成流程,为神经辐射场(NeRF)方法提供高效替代方案。通过利用显式3D表示的重建几何和FoundationStereo模型的深度估计,在专家知识迁移框架下获得有效几何信息。实验发现,将立体模型在3DGS生成的数据集上微调后,零样本泛化表现具有竞争力。直接使用重建几何时,噪声与伪影会传播至模型;而采用FoundationStereo的视差估计则更为干净,带来更好性能。本方法展示了低成本、高保真数据集构建及快速微调的潜力。此外,尽管最新高斯点云方法在标准基准上表现优异,其在真实复杂场景中的鲁棒性仍有待提升。

原文摘要 · Abstract (English)

In this paper, we introduce a 3D Gaussian Splatting (3DGS)-based pipeline for stereo dataset generation, offering an efficient alternative to Neural Radiance Fields (NeRF)-based methods. To obtain useful geometry estimates, we explore utilizing the reconstructed geometry from the explicit 3D representations as well as depth estimates from the FoundationStereo model in an expert knowledge transfer setup. We find that when fine-tuning stereo models on 3DGS-generated datasets, we demonstrate competitive performance in zero-shot generalization benchmarks. When using the reconstructed geometry directly, we observe that it is often noisy and contains artifacts, which propagate noise to the trained model. In contrast, we find that the disparity estimates from FoundationStereo are cleaner and consequently result in a better performance on the zero-shot generalization benchmarks. Our method highlights the potential for low-cost, high-fidelity dataset creation and fast fine-tuning for deep stereo models. Moreover, we also reveal that while the latest Gaussian Splatting based methods have achieved superior performance on established benchmarks, their robustness falls short in challenging in-the-wild settings warranting further exploration.

立体生成3D高斯数据合成深度估计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。