用线性注意力和一步扩散实现高效高精度单目深度估计
Efficient and High-Quality Depth Estimation via Pixel-Space Diffusion with Linear Attention

- 采用分层结构:先修复整体结构,再精细恢复边缘
- 在1080P下推理速度提升7.6倍,1440P下提升10.9倍
- 适合需要实时深度估计的视觉任务,如自动驾驶
本文提出Lapis,一种基于线性注意力的像素空间生成框架,实现高效且高保真的单目深度估计。尽管生成模型显著提升了深度估计的细节质量,但标准注意力的O(N²)复杂度和多步去噪过程导致高分辨率应用时计算成本过高。虽然线性注意力和一步预测在直觉上可行,但直接应用会导致结构不一致、细节丢失和噪声。Lapis通过粗到精的层级结构解决这些问题:首先利用补丁级一致性模块结合语义与空间先验恢复结构连贯性;随后通过基于跳跃连接的像素对应关系模块恢复锐利几何边界。此外,为缓解一步扩散固有的采样噪声,采用直接x-预测策略并利用流形假设瞄准干净数据流形。多个基准测试表明,Lapis在不同分辨率下均达到当前最优精度与边界清晰度,相较于以往最先进生成模型,在1080P下推理延迟降低7.6倍,1440P下降低10.9倍。
原文摘要 · Abstract (English)
This work presents $\textbf{Lapis}$, a $\textbf{l}$inear-$\textbf{a}$ttention-based $\textbf{pi}$xel-$\textbf{s}$pace generative framework that achieves efficient and high-fidelity depth estimation with one-step diffusion. While generative frameworks have significantly advanced monocular depth estimation with superior detail fidelity, the $\mathcal{O}(N^2)$ complexity of standard attention and the multi-step denoising process introduce prohibitive computational costs when scaling them to high-resolution image applications. Although linear attention and one-step prediction are intuitively viable, directly applying them leads to poor structural consistency, detail loss, and noise. Lapis rectifies these limitations through a coarse-to-fine hierarchy. Specifically, a Patch-level Consistency Module restores structural coherence by integrating semantic and spatial priors. Subsequently, a Pixel-level Refinement Module recovers sharp geometric boundaries via skip-connection-based pixel correspondence. Furthermore, to mitigate sampling noise inherent in one-step diffusion, we leverage the manifold assumption and adopt a direct $\mathbf{x}$-prediction strategy to target the clean data manifold. Extensive evaluations on multiple benchmarks demonstrate that Lapis consistently achieves state-of-the-art (SOTA) accuracy and boundary sharpness across various resolutions, reducing inference latency by up to 7.6$\times$ at 1080P and 10.9$\times$ at 1440P resolution compared to previous SOTA generative models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。