无需训练即可跨数据集推深度，提升真实场景泛化能力
Zero-Shot Depth from Defocus
- 设计基于Transformer的FOSSA网络，用聚焦距离嵌入增强多焦距信息交互
- 在新基准ZEDD上误差降低55.7%，显著优于现有方法
- 适配真实世界应用，适合追求零样本泛化的研究者
深度估计从模糊（Depth from Defocus, DfD）是从一系列不同焦距的图像中估计稠密度量深度图的任务。不同于以往方法对特定数据集过拟合，本文关注更具挑战性和实际意义的零样本泛化设置。我们首先提出一个全新的真实世界DfD基准ZEDD，其场景数量比先前基准多8.3倍，图像质量和真实深度图精度显著更高。同时，设计了一种名为FOSSA的新网络架构，为DfD任务量身定制，采用基于Transformer的结构与焦点距离嵌入的堆栈注意力层，实现跨焦距高效信息交换。此外，开发了一种新训练数据生成管道，利用现有大规模RGBD数据集合成仿真焦点堆栈。在ZEDD及其他基准上的实验表明，相比基线方法，误差最高降低55.7%。ZEDD基准已发布于https://zedd.cs.princeton.edu，代码与模型权重见https://github.com/princeton-vl/FOSSA。
原文摘要 · Abstract (English)
Depth from Defocus (DfD) is the task of estimating a dense metric depth map from a focus stack. Unlike previous works overfitting to a certain dataset, this paper focuses on the challenging and practical setting of zero-shot generalization. We first propose a new real-world DfD benchmark ZEDD, which contains 8.3x more scenes and significantly higher quality images and ground-truth depth maps compared to previous benchmarks. We also design a novel network architecture named FOSSA. FOSSA is a Transformer-based architecture with novel designs tailored to the DfD task. The key contribution is a stack attention layer with a focus distance embedding, allowing efficient information exchange across the focus stack. Finally, we develop a new training data pipeline allowing us to utilize existing large-scale RGBD datasets to generate synthetic focus stacks. Experiment results on ZEDD and other benchmarks show a significant improvement over the baselines, reducing errors by up to 55.7%. The ZEDD benchmark is released at https://zedd.cs.princeton.edu. The code and checkpoints are released at https://github.com/princeton-vl/FOSSA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。