解决透明物体深度估计难题，同时还原透明层与背景信息。
Multi-Label Stereo Matching for Transparent Scene Depth Estimation
- 用多标签回归建模同一像素多个深度值，突破单模态假设。
- 通过可迭代更新的多元高斯表示，精准识别需多标签的像素。
- 自建10场景89物体数据集，显著提升透明表面重建效果。
本文提出一种多标签立体匹配方法，用于同时估计透明场景中透明物体与被遮挡背景的深度。不同于以往假设视差维度为单峰分布、将匹配问题建模为单标签回归的方法，我们采用多标签回归框架，以在同一像素处估计多个深度值。为解决多标签回归问题，引入像素级多元高斯表示：均值向量编码同一像素的多个深度值，协方差矩阵判断是否需要多标签表征。该表示在GRU框架中迭代预测：每轮先预测均值参数的更新步长，再结合更新后的均值估计协方差矩阵。我们还构建了一个包含10个场景和89个物体的合成数据集，用于验证透明场景深度估计性能。实验表明，本方法在透明表面深度估计上显著提升，同时有效保留了背景信息，利于完整场景重建。代码已公开于https://github.com/BFZD233/TranScene。
原文摘要 · Abstract (English)
In this paper, we present a multi-label stereo matching method to simultaneously estimate the depth of the transparent objects and the occluded background in transparent scenes.Unlike previous methods that assume a unimodal distribution along the disparity dimension and formulate the matching as a single-label regression problem, we propose a multi-label regression formulation to estimate multiple depth values at the same pixel in transparent scenes. To resolve the multi-label regression problem, we introduce a pixel-wise multivariate Gaussian representation, where the mean vector encodes multiple depth values at the same pixel, and the covariance matrix determines whether a multi-label representation is necessary for a given pixel. The representation is iteratively predicted within a GRU framework. In each iteration, we first predict the update step for the mean parameters and then use both the update step and the updated mean parameters to estimate the covariance matrix. We also synthesize a dataset containing 10 scenes and 89 objects to validate the performance of transparent scene depth estimation. The experiments show that our method greatly improves the performance on transparent surfaces while preserving the background information for scene reconstruction. Code is available at https://github.com/BFZD233/TranScene.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。