用强化学习让图像生成模型提速3倍，质量不下降。
Accelerating Inference of Masked Image Generators via Reinforcement Learning
- 将加速生成问题转化为强化学习任务，设计质量与速度双奖励
- 在保持图像质量的前提下，推理步数减少至原来的1/3
- 适合需要快速生成高质量图像的场景，如实时应用
掩码生成模型（MGM）在生成高保真图像方面表现强劲，但需大量采样步骤，导致推理速度慢。本文提出Speed-RL新范式，通过强化学习加速预训练的MGM，在更少步骤内生成高质量图像。不同于传统知识蒸馏将加速问题视为分布匹配，我们将其建模为强化学习问题：结合图像质量奖励与推理速度奖励，以联合奖励为目标微调基础模型。大量实验表明，该方法可使模型推理速度提升3倍，同时保持与原模型相当的图像质量。
原文摘要 · Abstract (English)
Masked Generative Models (MGM)s demonstrate strong capabilities in generating high-fidelity images. However, they need many sampling steps to create high-quality generations, resulting in slow inference speed. In this work, we propose Speed-RL, a novel paradigm for accelerating a pretrained MGMs to generate high-quality images in fewer steps. Unlike conventional distillation methods which formulate the acceleration problem as a distribution matching problem, where a few-step student model is trained to match the distribution generated by a many-step teacher model, we consider this problem as a reinforcement learning problem. Since the goal of acceleration is to generate high quality images in fewer steps, we can combine a quality reward with a speed reward and finetune the base model using reinforcement learning with the combined reward as the optimization target. Through extensive experiments, we show that the proposed method was able to accelerate the base model by a factor of 3x while maintaining comparable image quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。