arXiv:2507.05604cs.CVeess.IV2025-07NeurIPS被引 4

通过粒子群密度搜索,提升图像修复的清晰度与稳定性。

Kernel Density Steering: Inference-Time Scaling via Mode Seeking for Image Restoration

  • 用多粒子扩散样本计算局部密度梯度,引导修复方向
  • 在真实世界超分辨率和修复任务中显著提升质量
  • 无需重训练,可直接接入现有扩散模型

扩散模型在图像修复中展现出潜力,但现有方法常面临保真度不一致和不良伪影问题。为此,我们提出核密度引导(Kernel Density Steering, KDS),一种新颖的推理时框架,通过显式局部模式搜索实现鲁棒、高保真的输出。KDS采用$N$个粒子组成的扩散样本集合,基于其整体输出计算局部核密度估计梯度,将每个粒子的图像块导向集合中识别出的高密度区域。这种集体局部模式搜索机制如同‘集体智慧’,使样本避开因独立采样或模型缺陷导致的虚假模式,转向更稳定、高质量的结构。该方法以更高计算开销换取更优样本质量,且作为即插即用框架,无需重新训练或外部验证器,可无缝集成至多种扩散采样器。大量数值验证表明,KDS在具有挑战性的现实世界超分辨率与图像修复任务中显著提升定量与定性性能。

原文摘要 · Abstract (English)

Diffusion models show promise for image restoration, but existing methods often struggle with inconsistent fidelity and undesirable artifacts. To address this, we introduce Kernel Density Steering (KDS), a novel inference-time framework promoting robust, high-fidelity outputs through explicit local mode-seeking. KDS employs an $N$-particle ensemble of diffusion samples, computing patch-wise kernel density estimation gradients from their collective outputs. These gradients steer patches in each particle towards shared, higher-density regions identified within the ensemble. This collective local mode-seeking mechanism, acting as "collective wisdom", steers samples away from spurious modes prone to artifacts, arising from independent sampling or model imperfections, and towards more robust, high-fidelity structures. This allows us to obtain better quality samples at the expense of higher compute by simultaneously sampling multiple particles. As a plug-and-play framework, KDS requires no retraining or external verifiers, seamlessly integrating with various diffusion samplers. Extensive numerical validations demonstrate KDS substantially improves both quantitative and qualitative performance on challenging real-world super-resolution and image inpainting tasks.

图像修复扩散模型推理优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。