用模型预测控制让生成模型高效解决图像逆问题。
Solving Inverse Problems with Flow-based Models via Model Predictive Control
- 将生成模型的逆问题求解转为分步控制子问题，无需反向传播。
- 在图像修复任务中表现优异，支持320亿参数大模型在消费级硬件运行。
- 无需训练，可适配各类生成模型，适合快速部署与高阶生成需求。
基于流的生成模型为逆问题提供了强大的无条件先验，但如何引导其进行条件生成仍具挑战。近期工作将无需训练的条件生成建模为最优控制问题；然而，求解由此产生的轨迹优化计算和内存开销巨大，需对流动力学进行微分或伴随求解。本文提出MPC-Flow，一种模型预测控制框架，将基于流模型的逆问题求解转化为一系列控制子问题，实现推理时可行的最优控制引导。我们提供了理论分析，揭示MPC-Flow与底层最优控制目标的关联，并展示不同算法选择可生成一系列引导算法，包括避免对生成模型轨迹进行反向传播的方案。我们在多个基准图像恢复任务上评估了MPC-Flow，涵盖线性和非线性场景，如补全、去模糊和超分辨率。结果表明，其性能强大且可扩展至大规模最先进的架构，在消费级硬件上实现了对量化后的FLUX.2（32B）的训练免费引导。
原文摘要 · Abstract (English)
Flow-based generative models provide strong unconditional priors for inverse problems, but guiding their dynamics for conditional generation remains challenging. Recent work casts training-free conditional generation in flow models as an optimal control problem; however, solving the resulting trajectory optimisation is computationally and memory intensive, requiring differentiation through the flow dynamics or adjoint solves. We propose MPC-Flow, a model predictive control framework that formulates inverse problem solving with flow-based generative models as a sequence of control sub-problems, enabling practical optimal control-based guidance at inference time. We provide theoretical analysis linking MPC-Flow to the underlying optimal control objective and show how different algorithmic choices yield a spectrum of guidance algorithms, including regimes that avoid backpropagation through the generative model trajectory. We evaluate MPC-Flow on benchmark image restoration tasks, spanning linear and non-linear settings such as in-painting, deblurring, and super-resolution, and demonstrate strong performance and scalability to massive state-of-the-art architectures via training-free guidance of FLUX.2 (32B) in a quantised setting on consumer hardware.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。