根据提示复杂度动态分配计算资源，实现图像生成质量与成本的最优平衡。
Cost-Aware Routing for Efficient Text-To-Image Generation
- 按提示复杂度自动选择不同计算量的生成模型或去噪步数
- 在COCO和DiffusionDB上平均质量优于单一模型
- 适合追求高效高质生成的部署场景
扩散模型通过迭代去噪过程可生成高质量图像，但其高保真度伴随高昂计算成本，源于固有的串行生成机制。本文提出一种成本感知路由框架，使每条提示的计算量可根据其复杂度动态调整。系统自动将提示路由至最合适的文生图生成函数，可能对应不同去噪步数的扩散模型，或独立的文生图模型。相比统一降本方法（如知识蒸馏、量化），该方法通过学习仅对少数复杂提示使用高成本方案（如100+步去噪），而对简单提示采用低成本方案（如小型蒸馏模型），实现最优质量-成本权衡。在COCO和DiffusionDB上的实验表明，通过学习路由到九个已训练好的文生图模型，该方法可实现平均质量超过任一单独模型的表现。代码已开源：https://github.com/winglicopy/CATImage。
原文摘要 · Abstract (English)
Diffusion models are well known for their ability to generate a high-fidelity image for an input prompt through an iterative denoising process. Unfortunately, the high fidelity also comes at a high computational cost due to the inherently sequential generative process. In this work, we seek to optimally balance quality and computational cost, and propose a framework to allow the amount of computation to vary for each prompt, depending on its complexity. Each prompt is automatically routed to the most appropriate text-to-image generation function, which may correspond to a distinct number of denoising steps of a diffusion model, or a disparate, independent text-to-image model. Unlike uniform cost reduction techniques (e.g., distillation, model quantization), our approach achieves the optimal trade-off by learning to reserve expensive choices (e.g., 100+ denoising steps) only for a few complex prompts, and employ more economical choices (e.g., small distilled model) for less sophisticated prompts. We empirically demonstrate on COCO and DiffusionDB that by learning to route to nine already-trained text-to-image models, our approach is able to deliver an average quality that is higher than that achievable by any of these models alone. Code is available at https://github.com/winglicopy/CATImage.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。