提出无噪声确定性扩散模型,提升密集预测精度与效率。
$\mathrm{D}^\mathrm{3}$-Predictor: Noise-Free Deterministic Diffusion for Dense Prediction
- 将预训练扩散模型重构成无随机噪声的确定性架构
- 单步推理,训练数据量减半,性能达顶尖水平
- 适合需要高精度几何结构的密集预测任务
尽管具有强视觉先验的扩散模型已成为强大的密集预测骨干,但其核心缺陷被忽视:扩散采样中的随机噪声与要求图像到几何的确定性映射的密集预测不匹配。本文表明,这种随机噪声会破坏细粒度空间线索,并使模型趋向于特定时间步的噪声目标,从而破坏有意义的几何结构映射。为此,我们提出D³-Predictor,一种基于预训练扩散模型重构的无噪声确定性扩散密集预测模型。该模型不再依赖含噪输入来利用扩散先验,而是将预训练扩散网络视为一系列时间步相关的视觉专家,通过自监督方式聚合其异构先验,形成单一、干净且完整的几何先验。同时,利用任务特定监督,无缝将此无噪声先验适配至密集预测任务。在多种密集预测任务上的大量实验表明,D³-Predictor在多样化场景中达到竞争性或领先性能。此外,其训练数据量不足此前的一半,且可实现单步高效推理。代码、数据与检查点已公开于 https://x-gengroup.github.io/HomePage_D3-Predictor/。
原文摘要 · Abstract (English)
Although diffusion models with strong visual priors have emerged as powerful dense prediction backbones, they overlook a core limitation: the stochastic noise at the core of diffusion sampling is inherently misaligned with dense prediction that requires a deterministic mapping from image to geometry. In this paper, we show that this stochastic noise corrupts fine-grained spatial cues and pushes the model toward timestep-specific noise objectives, consequently destroying meaningful geometric structure mappings. To address this, we introduce $\mathrm{D}^\mathrm{3}$-Predictor, a noise-free deterministic diffusion-based dense prediction model built by reformulating a pretrained diffusion model without stochasticity noise. Instead of relying on noisy inputs to leverage diffusion priors, $\mathrm{D}^\mathrm{3}$-Predictor views the pretrained diffusion network as an ensemble of timestep-dependent visual experts and self-supervisedly aggregates their heterogeneous priors into a single, clean, and complete geometric prior. Meanwhile, we utilize task-specific supervision to seamlessly adapt this noise-free prior to dense prediction tasks. Extensive experiments on various dense prediction tasks demonstrate that $\mathrm{D}^\mathrm{3}$-Predictor achieves competitive or state-of-the-art performance in diverse scenarios. In addition, it requires less than half the training data previously used and efficiently performs inference in a single step. Our code, data, and checkpoints are publicly available at https://x-gengroup.github.io/HomePage_D3-Predictor/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。