用混合数据训练立体匹配模型,显著提升跨场景泛化能力。
Stereo Anything: Unifying Zero-shot Stereo Matching with Large-Scale Mixed Data
- 融合真实标注数据与单目图像生成的合成数据,构建大规模混合训练集。
- 在四个公开基准上实现零样本泛化性能新纪录,跨域适应性更强。
- 无需修改模型结构,适合作为通用立体匹配解决方案推广使用。
立体匹配是3D视觉的核心任务,旨在建立双目图像间的像素级对应以恢复深度。尽管深度神经网络推动了显著进展,但现有模型在未见场景中常出现性能下降,主要源于训练数据多样性不足。本文提出StereoAnything,一种以数据为中心的框架,显著增强现有立体模型的零样本泛化能力。不设计新架构,而是通过系统整合两类异构数据源:(1) 覆盖多样化环境的标注数据集;(2) 由无标签单目图像生成的大规模合成双目图像对。该混合数据策略在跨域中提供一致且稳健的学习信号,有效缓解数据集偏差。在四个公开基准上的广泛零样本评估表明,Stereo Anything实现了领先的泛化性能。本工作为实现真正通用的立体匹配铺平道路,提供了一种可扩展的数据范式,适用于任何双目图像对。代码已开源:https://github.com/XiandaGuo/OpenStereo。
原文摘要 · Abstract (English)
Stereo matching serves as a cornerstone in 3D vision, aiming to establish pixel-wise correspondences between stereo image pairs for depth recovery. Despite remarkable progress driven by deep neural architectures, current models often exhibit severe performance degradation when deployed in unseen domains, primarily due to the limited diversity of training data. In this work, we introduce StereoAnything, a data-centric framework that substantially enhances the zero-shot generalization capability of existing stereo models. Rather than devising yet another specialized architecture, we scale stereo training to an unprecedented level by systematically unifying heterogeneous stereo sources: (1) curated labeled datasets covering diverse environments, and (2) large-scale synthetic stereo pairs generated from unlabeled monocular images. Our mixed-data strategy delivers consistent and robust learning signals across domains, effectively mitigating dataset bias. Extensive zero-shot evaluations on four public benchmarks demonstrate that Stereo Anything achieves state-of-the-art generalization. This work paves the way towards truly universal stereo matching, offering a scalable data paradigm applicable to any stereo image pair. We extensively evaluate the zero-shot capabilities of our model on four public datasets, showcasing its impressive ability to generalize to any stereo image pair. Code is available at https://github.com/XiandaGuo/OpenStereo.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。