arXiv:2510.07316cs.CV2025-10NeurIPS被引 51

直接在像素空间生成深度图,消除传统方法的飞边伪影。

Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformers

论文配图:Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformers
图 1 · 摘自论文原文
  • 采用像素空间扩散生成,避免VAE压缩引入的伪影。
  • 在五个基准上表现最优,边缘感知点云质量显著提升。
  • 适合需要高精度深度图的三维重建与机器人导航场景。

本文提出Pixel-Perfect Depth,一种基于像素空间扩散生成的单目深度估计模型,可从估计的深度图生成高质量、无飞边伪影的点云。现有生成式深度估计模型通过微调Stable Diffusion实现优异性能,但需使用VAE将深度图压缩至隐空间,不可避免地在边缘和细节处引入飞边伪影。本模型直接在像素空间进行扩散生成,避免了VAE带来的伪影。为应对像素空间生成带来的高复杂度,我们提出两项创新设计:1)语义提示扩散Transformer(SP-DiT),将视觉基础模型的语义表示融入DiT,以引导扩散过程,在保持全局语义一致性的同时增强细粒度视觉细节;2)级联DiT结构,逐步增加令牌数量,进一步提升效率与精度。模型在五个公开基准上达到所有已发表生成式模型的最佳性能,并在边缘感知点云评估中显著优于其他模型。

原文摘要 · Abstract (English)

This paper presents Pixel-Perfect Depth, a monocular depth estimation model based on pixel-space diffusion generation that produces high-quality, flying-pixel-free point clouds from estimated depth maps. Current generative depth estimation models fine-tune Stable Diffusion and achieve impressive performance. However, they require a VAE to compress depth maps into latent space, which inevitably introduces \textit{flying pixels} at edges and details. Our model addresses this challenge by directly performing diffusion generation in the pixel space, avoiding VAE-induced artifacts. To overcome the high complexity associated with pixel-space generation, we introduce two novel designs: 1) Semantics-Prompted Diffusion Transformers (SP-DiT), which incorporate semantic representations from vision foundation models into DiT to prompt the diffusion process, thereby preserving global semantic consistency while enhancing fine-grained visual details; and 2) Cascade DiT Design that progressively increases the number of tokens to further enhance efficiency and accuracy. Our model achieves the best performance among all published generative models across five benchmarks, and significantly outperforms all other models in edge-aware point cloud evaluation.

深度估计扩散模型三维重建图像生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。