arXiv:2511.19917cs.CV2025-11被引 8

不重新生成整图,只修复低质量区域,大幅节省算力。

Scale Where It Matters: Training-Free Localized Scaling for Diffusion Models

  • 通过对比注意力信号定位图像缺陷区域,无需训练即可实现精准局部重采样。
  • 在SD2.1、SDXL、FLUX上提升局部质量与全局一致性的同时,降低2-4倍显存开销。
  • 适合追求高效高质量生成的用户,尤其适用于资源受限场景。

扩散模型已成为文本到图像生成的主流范式,测试时缩放(TTS)通过在推理阶段增加计算量来提升样本质量。然而现有TTS方法对整图重采样,而生成质量常呈空间异质性,导致对已正确区域的冗余计算,且局部缺陷修正不足。本文提出新方向——局部化TTS,自适应地仅重采样缺陷区域并保留高质量部分,显著缩小搜索空间。这带来两大挑战:缺陷精确定位与全局一致性保持。我们提出LoTTS,首个完全免训练的局部化TTS框架。缺陷定位方面,LoTTS利用质量感知提示(如高质量与低质量)对比交叉注意力与自注意力信号,识别缺陷区域并细化为连贯掩码。一致性保障方面,仅扰动并局部去噪缺陷区域,确保修正范围可控,其余图像不受影响。在SD2.1、SDXL和FLUX上的大量实验表明,LoTTS达到当前最优性能:持续提升局部质量与全局保真度,同时相比Best-of-N采样降低2-4倍GPU成本。这些发现确立了局部化TTS作为推理阶段扩展扩散模型的有前景新方向。

原文摘要 · Abstract (English)

Diffusion models have become the dominant paradigm in text-to-image generation, and test-time scaling (TTS) improves sample quality by allocating additional computation at inference. Existing TTS methods, however, resample the entire image, while generation quality is often spatially heterogeneous. This leads to unnecessary computation on regions that are already correct, and localized defects remain insufficiently corrected. In this paper, we explore a new direction - Localized TTS - that adaptively resamples defective regions while preserving high-quality regions, thereby substantially reducing the search space. This raises two challenges: accurately localizing defects and maintaining global consistency. We propose LoTTS, the first fully training-free framework for localized TTS. For defect localization, LoTTS contrasts cross- and self-attention signals under quality-aware prompts (e.g., high-quality vs. low-quality) to identify defective regions, and then refines them into coherent masks. For consistency, LoTTS perturbs only defective regions and denoises them locally, ensuring that corrections remain confined while the rest of the image remains undisturbed. Extensive experiments on SD2.1, SDXL, and FLUX demonstrate that LoTTS achieves state-of-the-art performance: it consistently improves both local quality and global fidelity, while reducing GPU cost by 2-4x compared to Best-of-N sampling. These findings establish localized TTS as a promising new direction for scaling diffusion models at inference time.

扩散模型推理优化局部修复

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。