arXiv:2510.22010cs.CVcs.LG2025-10被引 3

无需反向传播,一键优化生成全过程,实现高效图像编辑。

FlowOpt: Fast Optimization Through Whole Flow Processes for Training-Free Editing

  • 将整个生成流程视为黑箱,零阶优化直接调控最终输出。
  • 在相同计算量下,编辑效果超越现有方法,支持中途监控与提前停止。
  • 理论保证收敛,适合需要快速调试的图像生成应用者。

扩散模型和流匹配模型的成功催生了大量测试时自适应技术,涵盖图像编辑、修复、压缩和个性化等任务。然而,由于采样过程具有迭代性,基于梯度的优化难以直接控制最终生成结果,现有方法通常仅在每一步单独操作。本文提出 FlowOpt——一种零阶(无梯度)优化框架,将整个生成流程视为黑箱,可在不反向传播的前提下对完整采样路径进行优化。该方法高效且支持实时监控中间结果,可实现早期停止。我们证明了在特定步长条件下,算法能收敛至全局最优,并提出了经验估计该上界的策略以选择合适步长。实验表明,FlowOpt可用于图像编辑,包括:(i) 反演(确定生成给定图像的初始噪声),(ii) 直接引导编辑图像接近源图并满足目标文本提示。两种场景下均达到当前最佳性能,且神经函数评估次数(NFEs)与现有方法相当。代码与示例已在项目主页公开。

原文摘要 · Abstract (English)

The remarkable success of diffusion and flow-matching models has ignited a surge of works on adapting them at test time for controlled generation tasks. Examples range from image editing to restoration, compression and personalization. However, due to the iterative nature of the sampling process in those models, it is computationally impractical to use gradient-based optimization to directly control the image generated at the end of the process. As a result, existing methods typically resort to manipulating each timestep separately. Here we introduce FlowOpt - a zero-order (gradient-free) optimization framework that treats the entire flow process as a black box, enabling optimization through the whole sampling path without backpropagation through the model. Our method is both highly efficient and allows users to monitor the intermediate optimization results and perform early stopping if desired. We prove a sufficient condition on FlowOpt's step-size, under which convergence to the global optimum is guaranteed. We further show how to empirically estimate this upper bound so as to choose an appropriate step-size. We demonstrate how FlowOpt can be used for image editing, showcasing two options: (i) inversion (determining the initial noise that generates a given image), and (ii) directly steering the edited image to be similar to the source image while conforming to a target text prompt. In both cases, FlowOpt achieves state-of-the-art results while using roughly the same number of neural function evaluations (NFEs) as existing methods. Code and examples are available on the project's webpage.

图像编辑零阶优化生成模型扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。