用自评估生成法,低成本构建真实世界光学流与立体匹配数据集
Self-Assessed Generation: Trustworthy Label Generation for Optical Flow and Stereo Matching in Real-world
- 基于重建场生成数据,无需人工标注
- 多维度量化置信度,消除生成缺陷
- 适合提升自监督模型在真实场景的泛化能力
当前光学流与立体匹配方法在真实世界中泛化能力差,主要因数据集制作成本高,且现有自监督方法存在结果模糊、训练复杂等问题。为此,我们提出统一的自监督泛化框架Self-Assessed Generation(SAG)。SAG为数据驱动,利用先进重建技术从RGB图像构建重建场,并据此生成数据集。随后,从重建场分布、几何一致性、结构相似性等多角度量化生成结果置信度,以消除生成过程中的固有缺陷。SAG还设计了3D飞行前景自动渲染流水线,促使网络学习遮挡和运动前景。实验表明,由于SAG不改变模型或损失函数,可直接用于训练最先进深度网络,显著提升自监督方法在主流光学流与立体匹配数据集上的泛化性能。相比以往训练模式,SAG更具通用性、成本效益与准确性。
原文摘要 · Abstract (English)
A significant challenge facing current optical flow and stereo methods is the difficulty in generalizing them well to the real world. This is mainly due to the high costs required to produce datasets, and the limitations of existing self-supervised methods on fuzzy results and complex model training problems. To address the above challenges, we propose a unified self-supervised generalization framework for optical flow and stereo tasks: Self-Assessed Generation (SAG). Unlike previous self-supervised methods, SAG is data-driven, using advanced reconstruction techniques to construct a reconstruction field from RGB images and generate datasets based on it. Afterward, we quantified the confidence level of the generated results from multiple perspectives, such as reconstruction field distribution, geometric consistency, and structural similarity, to eliminate inevitable defects in the generation process. We also designed a 3D flight foreground automatic rendering pipeline in SAG to encourage the network to learn occlusion and motion foreground. Experimentally, because SAG does not involve changes to methods or loss functions, it can directly self-supervised train the state-of-the-art deep networks, greatly improving the generalization performance of self-supervised methods on current mainstream optical flow and stereo-matching datasets. Compared to previous training modes, SAG is more generalized, cost-effective, and accurate.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。