提出双向对齐机制,提升立体视频匹配的时序一致性。
Match Stereo Videos via Bidirectional Alignment
- 用双向帧间对齐替代滑动窗口,减少时间抖动。
- 在多个基准上达到最新性能,尤其在户外场景更优。
- 新增真实与合成自然场景数据集,推动领域发展。
视频立体匹配旨在从校正后的立体视频中估计一致的视差图。当前方法在数据集和算法上均有提升空间。基于学习的方法多关注单对立体图像的性能优化,导致视频中存在时序不一致问题;现有视频方法通常采用时间维度的滑动窗口操作,易产生与窗口尺寸相关的低频振荡。为此,我们提出一种相邻帧间的双向对齐机制作为基础操作。基于此,构建了新型视频处理框架 BiDAStereo 及兼容通用图像方法的插件式稳定化网络 BiDAStabilizer。针对数据集,现有合成物体类与室内数据集为主,缺乏室外自然场景数据。为此,我们提出一个聚焦自然场景的真实合成数据集与基准,并采集了一个涵盖多样城市环境的真实立体视频数据集用于定性评估。在域内、域外及鲁棒性测试中,实验充分验证了方法与数据集的有效性,显著提升了预测质量,在多个常用基准上达到当前最优结果。项目主页、演示、代码与数据集见:https://tomtomtommi.github.io/BiDAVideo/
原文摘要 · Abstract (English)
Video stereo matching is the task of estimating consistent disparity maps from rectified stereo videos. There is considerable scope for improvement in both datasets and methods within this area. Recent learning-based methods often focus on optimizing performance for independent stereo pairs, leading to temporal inconsistencies in videos. Existing video methods typically employ sliding window operation over time dimension, which can result in low-frequency oscillations corresponding to the window size. To address these challenges, we propose a bidirectional alignment mechanism for adjacent frames as a fundamental operation. Building on this, we introduce a novel video processing framework, BiDAStereo, and a plugin stabilizer network, BiDAStabilizer, compatible with general image-based methods. Regarding datasets, current synthetic object-based and indoor datasets are commonly used for training and benchmarking, with a lack of outdoor nature scenarios. To bridge this gap, we present a realistic synthetic dataset and benchmark focused on natural scenes, along with a real-world dataset captured by a stereo camera in diverse urban scenes for qualitative evaluation. Extensive experiments on in-domain, out-of-domain, and robustness evaluation demonstrate the contribution of our methods and datasets, showcasing improvements in prediction quality and achieving state-of-the-art results on various commonly used benchmarks. The project page, demos, code, and datasets are available at: https://tomtomtommi.github.io/BiDAVideo/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。