arXiv:2501.08654cs.CV2025-01ICCV被引 14

用单图生成立体图,零样本匹配效果领先。

ZeroStereo: Zero-shot Stereo Matching from Single Images

  • 用单目深度模型生成伪视差,合成右图。
  • 微调扩散模型修复遮挡区,保留语义结构。
  • 无需训练即可提升伪标签可靠性,适合实际场景应用。

当前最先进的监督式立体匹配方法在多个基准测试中表现优异,但因真实世界立体数据标注稀缺,其在现实场景中的泛化能力仍受限。本文提出ZeroStereo,一种用于零样本立体匹配的新型图像生成流程。该方法利用单目深度估计模型生成的伪视差,将任意单图合成高质量右视图。与以往通过邻近像素或随机背景填充遮挡区域的方法不同,我们微调扩散图像修复模型以恢复缺失细节,同时保持语义一致性。此外,提出无训练置信度生成机制,无需额外训练即可缓解不可靠伪标签的影响;并设计自适应视差选择策略,确保视差分布多样且真实,避免过度遮挡和前景失真。实验表明,仅使用相当于Scene Flow规模的数据量,基于本流程训练的模型在多个数据集上实现领先的零样本泛化性能。代码已开源:https://github.com/Windsrain/ZeroStereo。

原文摘要 · Abstract (English)

State-of-the-art supervised stereo matching methods have achieved remarkable performance on various benchmarks. However, their generalization to real-world scenarios remains challenging due to the scarcity of annotated real-world stereo data. In this paper, we propose ZeroStereo, a novel stereo image generation pipeline for zero-shot stereo matching. Our approach synthesizes high-quality right images from arbitrary single images by leveraging pseudo disparities generated by a monocular depth estimation model. Unlike previous methods that address occluded regions by filling missing areas with neighboring pixels or random backgrounds, we fine-tune a diffusion inpainting model to recover missing details while preserving semantic structure. Additionally, we propose Training-Free Confidence Generation, which mitigates the impact of unreliable pseudo labels without additional training, and Adaptive Disparity Selection, which ensures a diverse and realistic disparity distribution while preventing excessive occlusion and foreground distortion. Experiments demonstrate that models trained with our pipeline achieve state-of-the-art zero-shot generalization across multiple datasets with only a dataset volume comparable to Scene Flow. Code: https://github.com/Windsrain/ZeroStereo.

立体匹配零样本图像生成扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。