让AI绘画又快又好,一套新方法通用多模型
CoRe^2: Collect, Reflect and Refine to Generate Better and Faster
- 三步走:收集轨迹、训练弱模型反射特征、强化生成
- 采样效率提升一倍,图像质量显著优于基线模型
- 适配扩散与自回归模型,适合追求高效高质的开发者
让文本到图像生成模型既快速又高质量是一个重要方向。以往方法通常在视觉质量和采样效率间取舍,且多数推理方法无法同时在扩散模型(DMs)和视觉自回归模型(ARMs)上保持稳定性能。本文提出一种全新的即插即用推理范式 CoRe^2,包含三个步骤:收集无分类器引导(CFG)轨迹,利用数据训练一个弱模型以反映易学内容,并将函数评估次数减少一半;随后通过弱到强引导对条件输出进行精细化生成,提升模型对高频和真实感内容的捕捉能力。CoRe^2 是首个在多种扩散模型(如 SDXL、SD3.5、FLUX)和自回归模型(如 LlamaGen)上同时实现高效与有效的方法,在 HPD v2、Pick-of-Pic、Drawbench、GenEval、T2I-Compbench 等多个评测中表现优异。还可无缝集成 Z-Sampling,使用 SD3.5 时在 PickScore 与 AES 上分别超越其 0.3 和 0.16,节省 5.64 秒时间。代码已开源。
原文摘要 · Abstract (English)
Making text-to-image (T2I) generative model sample both fast and well represents a promising research direction. Previous studies have typically focused on either enhancing the visual quality of synthesized images at the expense of sampling efficiency or dramatically accelerating sampling without improving the base model's generative capacity. Moreover, nearly all inference methods have not been able to ensure stable performance simultaneously on both diffusion models (DMs) and visual autoregressive models (ARMs). In this paper, we introduce a novel plug-and-play inference paradigm, CoRe^2, which comprises three subprocesses: Collect, Reflect, and Refine. CoRe^2 first collects classifier-free guidance (CFG) trajectories, and then use collected data to train a weak model that reflects the easy-to-learn contents while reducing number of function evaluations during inference by half. Subsequently, CoRe^2 employs weak-to-strong guidance to refine the conditional output, thereby improving the model's capacity to generate high-frequency and realistic content, which is difficult for the base model to capture. To the best of our knowledge, CoRe^2 is the first to demonstrate both efficiency and effectiveness across a wide range of DMs, including SDXL, SD3.5, and FLUX, as well as ARMs like LlamaGen. It has exhibited significant performance improvements on HPD v2, Pick-of-Pic, Drawbench, GenEval, and T2I-Compbench. Furthermore, CoRe^2 can be seamlessly integrated with the state-of-the-art Z-Sampling, outperforming it by 0.3 and 0.16 on PickScore and AES, while achieving 5.64s time saving using SD3.5.Code is released at https://github.com/xie-lab-ml/CoRe/tree/main.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。