无需训练,通过噪声对齐提升扩散模型的域适应能力
FreeDNA: Endowing Domain Adaptation of Diffusion-Based Dense Prediction with Training-Free Domain Noise Alignment
- 提出无训练域噪声对齐方法,利用噪声统计差异实现域适应
- 在四个密集预测任务上显著提升模型跨域性能
- 适用于有源域或无源域场景,尤其适合资源受限部署
密集预测任务的领域适应(DA)是重要研究方向,可提升模型在未见领域上的表现。随着基于扩散模型的密集预测(DDP)发展,针对该框架设计专门的域适应方法值得探索,因扩散模型擅长建模包含领域信息的分布变换。本文提出一种无需训练的机制,赋予DDP框架域适应能力。观察到扩散过程中的暴露偏差(如噪声统计偏差)会引入域偏移,且不同域在DDP模型下的表现可通过噪声预测统计有效捕捉。基于此,提出训练免费的域噪声对齐(DNA)方法,在采样过程中对齐目标域与源域的噪声统计,缓解其随域变化的差异。当源域可用时,直接用DNA对齐两域噪声统计;在更难的无源域场景中,利用采样过程中高置信区域的统计信息逐步引导噪声调整。实验表明,该方法在四个常见密集预测任务中均有效增强DDP模型的域适应能力。代码已开源。
原文摘要 · Abstract (English)
Domain Adaptation(DA) for dense prediction tasks is an important topic, which enhances the dense prediction model's performance when tested on its unseen domain. Recently, with the development of Diffusion-based Dense Prediction (DDP) models, the exploration of DA designs tailored to this framework is worth exploring, since the diffusion model is effective in modeling the distribution transformation that comprises domain information. In this work, we propose a training-free mechanism for DDP frameworks, endowing them with DA capabilities. Our motivation arises from the observation that the exposure bias (e.g., noise statistics bias) in diffusion brings domain shift, and different domains in conditions of DDP models can also be effectively captured by the noise prediction statistics. Based on this, we propose a training-free Domain Noise Alignment (DNA) approach, which alleviates the variations of noise statistics to domain changes during the diffusion sampling process, thereby achieving domain adaptation. Specifically, when the source domain is available, we directly adopt the DNA method to achieve domain adaptation by aligning the noise statistics of the target domain with those of the source domain. For the more challenging source-free DA, inspired by the observation that regions closer to the source domain exhibit higher confidence meeting variations of sampling noise, we utilize the statistics from the high-confidence regions progressively to guide the noise statistic adjustment during the sampling process. Notably, our method demonstrates the effectiveness of enhancing the DA capability of DDP models across four common dense prediction tasks. Code is available at \href{https://github.com/xuhang07/FreeDNA}{https://github.com/xuhang07/FreeDNA}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。