利用扩散模型的语义结构提升图像分类器的测试时适应能力
Exploring Structured Semantic Priors Underlying Diffusion Score for Test-time Adaptation
- 从单步去噪中提取扩散得分的结构化语义先验
- 在多种测试场景下显著提升预训练分类器性能
- 无需多步采样，适合高效部署的测试时自适应
融合生成与判别模型的优势一直是机器学习的重要目标。本文揭示了基于得分的生成模型中隐藏的语义结构，表明其可作为有效的判别先验。受理论发现启发，我们提出DUSA方法，利用扩散得分中的结构化语义先验，实现图像分类器或密集预测器的测试时自适应。DUSA仅需单步去噪过程即可提取知识，避免了传统蒙特卡洛似然估计对多步迭代的依赖。我们在多种测试场景下验证了DUSA对各类先进预训练模型的有效性，并通过详尽消融实验分析了关键组件。代码已公开于https://github.com/BIT-DA/DUSA。
原文摘要 · Abstract (English)
Capitalizing on the complementary advantages of generative and discriminative models has always been a compelling vision in machine learning, backed by a growing body of research. This work discloses the hidden semantic structure within score-based generative models, unveiling their potential as effective discriminative priors. Inspired by our theoretical findings, we propose DUSA to exploit the structured semantic priors underlying diffusion score to facilitate the test-time adaptation of image classifiers or dense predictors. Notably, DUSA extracts knowledge from a single timestep of denoising diffusion, lifting the curse of Monte Carlo-based likelihood estimation over timesteps. We demonstrate the efficacy of our DUSA in adapting a wide variety of competitive pre-trained discriminative models on diverse test-time scenarios. Additionally, a thorough ablation study is conducted to dissect the pivotal elements in DUSA. Code is publicly available at https://github.com/BIT-DA/DUSA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。