解决扩散模型反演中纹理丢失问题，提升图像生成质量。
Spectral Collapse in Diffusion Inversion
- 提出正交方差引导，修正反演过程中的噪声动力学
- 在显微镜超分辨率和素描转图像任务中恢复真实纹理
- 适合需要高保真结构与纹理的图像生成研究者
条件扩散反演为无配对图像到图像转换提供了强大框架。然而，我们通过详尽分析发现，当源域相比目标域频谱稀疏时（如超分辨率、素描转图像），标准确定性反演（如DDIM）会失效。此时，从输入恢复的潜在表示不遵循预期的各向同性高斯分布，而是呈现低频信号，导致目标采样生成结果过平滑、缺乏纹理。我们称此现象为频谱坍塌。尝试通过随机化方法恢复噪声方差的替代方案往往破坏与输入的语义关联，引发结构漂移。为解决这一结构-纹理权衡问题，我们提出一种推理时方法——正交方差引导（OVG），通过修正常微分方程动态，使噪声幅度在结构梯度零空间内符合理论高斯值。在显微镜超分辨率（BBBC021）和素描转图像（Edges2Shoes）上的大量实验表明，OVG能有效恢复照片级真实感纹理，同时保持结构保真度。
原文摘要 · Abstract (English)
Conditional diffusion inversion provides a powerful framework for unpaired image-to-image translation. However, we demonstrate through an extensive analysis that standard deterministic inversion (e.g. DDIM) fails when the source domain is spectrally sparse compared to the target domain (e.g., super-resolution, sketch-to-image). In these contexts, the recovered latent from the input does not follow the expected isotropic Gaussian distribution. Instead it exhibits a signal with lower frequencies, locking target sampling to oversmoothed and texture-poor generations. We term this phenomenon spectral collapse. We observe that stochastic alternatives attempting to restore the noise variance tend to break the semantic link to the input, leading to structural drift. To resolve this structure-texture trade-off, we propose Orthogonal Variance Guidance (OVG), an inference-time method that corrects the ODE dynamics to enforce the theoretical Gaussian noise magnitude within the null-space of the structural gradient. Extensive experiments on microscopy super-resolution (BBBC021) and sketch-to-image (Edges2Shoes) demonstrate that OVG effectively restores photorealistic textures while preserving structural fidelity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。