针对人像传输中的模糊与噪声问题,提出一步式扩散模型增强修复效果。
HAODiff: Human-Aware One-Step Diffusion via Dual-Prompt Guidance
- 设计双提示引导机制,利用高质量图、残差噪声和运动掩码训练。
- 在合成与真实数据上均超越当前最佳方法,峰值指标提升显著。
- 适合图像修复、视频去模糊及含人像场景的生成任务使用。
以人为核心的图像在传输过程中常面临严重通用退化与人体运动模糊(HMB)的双重挑战,现有研究对此关注不足。为此,我们构建了一个模拟二者共存的退化流程,生成合成退化数据以训练提出的HAODiff——一种面向人类的一步扩散修复模型。该模型采用三分支双提示引导(DPG)机制,分别以高质量图像、低质量与高质量之差(残差噪声)以及人体运动模糊分割掩码为训练目标,生成分类器无指引(CFG)所需的正负提示对,实现单步扩散下的自适应提示调整,显著提升对多种退化的鲁棒性。为公平评估,我们引入包含丰富噪声与运动模糊组合的MPII-Test基准。大量实验表明,无论在定量指标还是视觉质量上,本方法在合成与真实数据集(包括新提出的MPII-Test)中均优于现有最先进(SOTA)方法。代码已开源:https://github.com/gobunu/HAODiff。
原文摘要 · Abstract (English)
Human-centered images often suffer from severe generic degradation during transmission and are prone to human motion blur (HMB), making restoration challenging. Existing research lacks sufficient focus on these issues, as both problems often coexist in practice. To address this, we design a degradation pipeline that simulates the coexistence of HMB and generic noise, generating synthetic degraded data to train our proposed HAODiff, a human-aware one-step diffusion. Specifically, we propose a triple-branch dual-prompt guidance (DPG), which leverages high-quality images, residual noise (LQ minus HQ), and HMB segmentation masks as training targets. It produces a positive-negative prompt pair for classifier-free guidance (CFG) in a single diffusion step. The resulting adaptive dual prompts let HAODiff exploit CFG more effectively, boosting robustness against diverse degradations. For fair evaluation, we introduce MPII-Test, a benchmark rich in combined noise and HMB cases. Extensive experiments show that our HAODiff surpasses existing state-of-the-art (SOTA) methods in terms of both quantitative metrics and visual quality on synthetic and real-world datasets, including our introduced MPII-Test. Code is available at: https://github.com/gobunu/HAODiff.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。