arXiv:2607.04691cs.CV2026-07中稿 · ECCV

用闭环控制提升图像生成对参考图的保真度,无需重新训练。

From Open Loop to Closed Loop: A Test-Time Iterative Optimization Framework for Reference-Consistent Image Generation

论文配图:From Open Loop to Closed Loop: A Test-Time Iterative Optimization Framework for Reference-Consistent Image Generation
图 1 · 摘自论文原文
  • 测试时迭代优化潜在控制信号,通过反馈纠正生成偏差。
  • 在人脸、姿态、深度生成上分别提升25.36%、27.71%、28.50%精度。
  • 无需训练,适配现有扩散模型,适合需要高保真控制的场景。

可控图像生成虽已取得进展,但现有方法多为开环系统,仅单向注入控制信号,缺乏反馈机制,难以保证与参考图严格一致。为此,本文提出一种全新的测试时迭代优化框架,将参考一致性生成重构为闭环动态追踪问题。将预训练生成模型视为控制对象,采用基于改进比例-积分-微分(PID)算法的传感器-控制器架构,在测试阶段根据生成结果与参考目标的差异,迭代优化潜在控制信号。该方法完全无需训练、模型无关,可无缝集成至现有扩散流程。在身份保持、姿态控制和深度控制任务上的大量实验验证了其通用性。结果表明,相比计算量相当的开环基线,本方法在面部相似度上提升最高达25.36%,姿态对齐空间误差降低27.71%,深度一致性误差降低28.50%。更广泛地,本工作引入经典控制理论视角,证明可控生成可被有效建模为动态反馈系统,为生成模型优化提供了新范式。代码已开源。

原文摘要 · Abstract (English)

While controllable image generation has made significant strides by incorporating visual reference conditions, existing methods predominantly operate as open-loop systems. They inject control signals in a strictly feed-forward manner, failing to guarantee strict fidelity to the reference due to the absence of active feedback and error correction mechanisms. To address this fundamental limitation, we propose a novel test-time iterative optimization framework that reformulates reference-consistent generation as a closed-loop dynamic tracking problem. By treating the pre-trained generative model as a control plant, our framework employs a sensor-controller architecture driven by a modified Proportional-Integral-Derivative (PID) algorithm. This mechanism iteratively optimizes the latent control signals at test time based on the sensed discrepancy between the generated output and the reference target. Notably, this approach is entirely training-free, model-agnostic, and integrates seamlessly around existing diffusion pipelines. Extensive evaluations across ID-preserving, pose-controlled, and depth-controlled generation tasks validate the universality of our method. Empirical results demonstrate improvements over computation-matched open-loop baselines, achieving relative performance gains of up to 25.36\% for facial similarity, alongside spatial error reductions of up to 27.71\% for pose alignment and 28.50\% for depth consistency. More broadly, this work offers a new conceptual perspective: it demonstrates that controllable generation can be effectively managed as a dynamic feedback system, bringing the rigorous principles of classical control theory into the optimization of generative models. Code is available at https://github.com/zzdrill/From-Open-Loop-to-Closed-Loop.

图像生成闭环控制扩散模型测试优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。