提出新基准测试负视差,发现现有模型在视差穿越零平面时性能暴跌。
Disparity Has a Sign: Stereo Matching Beyond the Zero-Disparity Plane

- 设计新基准ZDPShift,含21495对带符号真值的立体图像。
- 模型在负视差区域EPE飙升至75.33像素,超3像素误差像素占比过半。
- 预训练特征本可处理负视差,仅因输出约定导致失效,适合改进立体匹配系统者阅读。
现代立体匹配模型在视差穿越零视差平面(ZDP)时表现严重下降,端点误差(EPE)提升4.6至37倍。然而从影院3D到VR的内容中,大量物体位于ZDP后方,对应负视差。当前数据集、架构与评估协议均默认非负几何,因校正平行相机将ZDP置于无穷远,导致所有有限深度视差d=fB/z≥0,标准流程无法测量或违反负视差。为此,本文提出ZDPShift基准,包含七部导演创作的公开电影帧,每帧在五个不同ZDP位置渲染,共21,495对立体图像,附密集符号真值。六种先进图像与视频立体匹配模型在平面移动后全部崩溃:FoundationStereo的EPE从2.24像素升至75.33像素,所有骨干网络均导致约一半像素误差超过3像素。但问题并非匹配能力缺失——在仅用SceneFlow合成监督训练时,错误在正负区间保持稳定。仅训练解码器而冻结预训练匹配特征,六种骨干网络均表现良好,EPE波动小于0.2像素。表明预训练特征本身已具备负视差处理能力,仅因输出惯例被丢弃。同时,其在KITTI、Middlebury、ETH3D和Sintel上的正视差精度基本保留。
原文摘要 · Abstract (English)
Modern stereo matching models fail when disparity crosses zero, with end-point error (EPE) rising by 4.6-37$\times$. Yet stereoscopic content, from cinema 3D to VR, routinely contains objects behind the zero-disparity plane (ZDP), corresponding to negative disparities. The blind spot cascades through datasets, architectures, and evaluation protocols, all of which inherit the non-negative geometry. Rectified parallel cameras place ZDP at infinity, so every finite depth yields $d=fB/z \ge 0$ by construction, and nothing within the standard pipeline can violate, or even measure, a negative disparity. To measure it, we propose \textit{ZDPShift}, a benchmark of $21{,}495$ stereo pairs from seven cinematographer-authored open movies, each frame rendered at five zero-disparity-plane positions with dense signed ground truth. Six state-of-the-art image and video stereo matching models collapse once the plane moves. On identical scene content, FoundationStereo goes from $2.24$ px EPE to $75.33$px, with every backbone leaving roughly half of all pixels exceeding a three-pixel disparity error. What is missing, however, is not the underlying matching capability. % The capability itself, however, is already present. Training on supervision synthesized from SceneFlow, which adds no new data or parameters, keeps the error flat across the signed range. Training only the decoder, with the pretrained matching features frozen, performs comparably across all six backbones, with EPE jittering within $0.2$px. Thus, the pretrained features already extend to the negative regime they were never trained on, and only the output convention discarded it. Meanwhile, positive-regime accuracy on KITTI, Middlebury, ETH3D, and Sintel is largely preserved.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。