用单步确定性方法提升图像修复任务一致性
Noise-Free One-Step LoRA for Task-Driven Image Restoration with Diffusion Priors
- 采用预训练扩散模型的单步前向传播，避免随机噪声干扰
- LoRA适配模块使修复后任务准确率显著提升，超越多步基线
- 适合需要快速高保真修复的视觉任务应用
退化图像不仅降低视觉质量，还影响高层视觉任务性能。任务驱动图像修复（TDIR）通过联合优化修复质量与任务表现来应对该问题。近期研究显示，预训练扩散先验有助于提升TDIR效果，但基于扩散的方法本质具有随机性，采样过程依赖随机噪声，可能破坏任务一致性。本文表明，使用预训练扩散先验的确定性、无噪声单步前向传播可显著提升TDIR性能，但其效果高度依赖适配模块：LoRA带来稳定增益，而类似ControlNet的条件机制则无效。这使得单步前向即可超越传统多步扩散TDIR基线。此外，我们提出一种保持任务性能的GAN训练策略，提升感知质量而不损失任务表现。在分类、分割和检测任务上的大量实验验证了持续优势，并进一步在真实退化图像和OCR任务上验证了泛化能力。
原文摘要 · Abstract (English)
Degraded images not only reduce visual quality but also impair downstream high-level vision tasks. Task-driven image restoration (TDIR) addresses this issue by jointly optimizing restoration quality and task performance. Recent works show that pretrained diffusion priors benefit TDIR, yet diffusion-based restoration is inherently stochastic, as the sampling process depends on a random noise term, which can undermine task consistency. In this paper, we show that a deterministic, noise-free one-step forward pass with pretrained diffusion priors can substantially improve TDIR, but the benefit critically depends on the adaptation module: LoRA yields consistent gains, whereas ControlNet-style conditioning does not. This enables one-step forwarding that surpasses conventional multi-step diffusion TDIR baselines. Furthermore, we introduce a task-preserving GAN training strategy that improves perceptual quality without sacrificing task performance. Extensive experiments on classification, segmentation, and detection demonstrate consistent gains over prior TDIR methods, and we further validate generalization on real-world degraded images and OCR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。