用暗知识建模多模态真实分布,提升立体匹配跨域泛化能力
MIDAS: Modeling Ground-Truth Distributions with Dark Knowledge for Domain Generalized Stereo Matching
- 从预训练网络提取相似性与不确定性暗知识,建模边缘与非边缘区域的真实分布
- 通过集成学习区分客观与偏见知识,在拉普拉斯参数空间实现联合建模
- 显著提升模型在真实场景下的跨域性能,适合复杂环境下的立体匹配应用
尽管域泛化立体匹配已取得显著进展,现有方法在从合成数据向真实数据迁移时仍表现出域偏好,限制了其在复杂多变场景中的实际应用。立体网络预测的概率分布天然包含丰富的相似性和不确定性信息。受此启发,我们提出从预训练网络中提取这两类暗知识,以建模边缘与非边缘区域的直观多模态真实分布。为缓解单个网络的固有域偏好,采用网络集成,并在拉普拉斯参数空间区分客观与偏差知识。最终,将客观知识与原始视差标签联合建模为拉普拉斯混合模型,为立体网络训练提供细粒度监督。大量实验表明:(1) 本方法具有通用性,可有效提升现有网络的泛化能力;(2) 配合PCWNet在KITTI 2015和2012数据集上达到当前最优泛化性能;(3) 在四个主流真实数据集上的综合排名优于现有方法。
原文摘要 · Abstract (English)
Despite the significant advances in domain generalized stereo matching, existing methods still exhibit domain-specific preferences when transferring from synthetic to real domains, hindering their practical applications in complex and diverse scenarios. The probability distributions predicted by the stereo network naturally encode rich similarity and uncertainty information. Inspired by this observation, we propose to extract these two types of dark knowledge from the pre-trained network to model intuitive multi-modal ground-truth distributions for both edge and non-edge regions. To mitigate the inherent domain preferences of a single network, we adopt network ensemble and further distinguish between objective and biased knowledge in the Laplace parameter space. Finally, the objective knowledge and the original disparity labels are jointly modeled as a mixture of Laplacians to provide fine-grained supervision for the stereo network training. Extensive experiments demonstrate that: (1) Our method is generic and effectively improves the generalization of existing networks. (2) PCWNet with our method achieves the state-of-the-art generalization performance on both KITTI 2015 and 2012 datasets. (3) Our method outperforms existing methods in comprehensive ranking across four popular real-world datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。