Argus智能调度文本生成图像模型，平衡速度与质量。
Argus: Quality-Aware High-Throughput Text-to-Image Inference Serving System
- 按提示词动态选择不同近似策略，智能匹配模型
- 相比基线延迟违规减少10倍，吞吐量提升40%
- 适合需要高并发且对图像质量有要求的场景
文本到图像（T2I）模型广受欢迎，多数为扩散模型，具有高度计算密集型特点，依赖迭代去噪过程生成图像，导致推理延迟极高。这给构建高吞吐系统带来挑战。我们发现大量提示词可使用更快的近似模型服务，但需针对每条提示精确校准近似程度以避免质量下降。如何在固定规模集群上为每个提示分配合适模型及兼容的近似设置，仍是难题。本文提出Argus，一个高质量、高吞吐的T2I推理系统，通过为每条提示选择恰当的近似层级，在满足吞吐目标的同时保障图像质量。Argus智能切换不同近似策略，兼顾性能与输出质量。在两个真实工作负载数据集上，相比基线系统，其延迟服务等级目标（SLO）违反次数减少10倍，平均质量提升10%，吞吐量提高40%。
原文摘要 · Abstract (English)
Text-to-image (T2I) models have gained significant popularity. Most of these are diffusion models with unique computational characteristics, distinct from both traditional small-scale ML models and large language models. They are highly compute-bound and use an iterative denoising process to generate images, leading to very high inference time. This creates significant challenges in designing a high-throughput system. We discovered that a large fraction of prompts can be served using faster, approximated models. However, the approximation setting must be carefully calibrated for each prompt to avoid quality degradation. Designing a high-throughput system that assigns each prompt to the appropriate model and compatible approximation setting remains a challenging problem. We present Argus, a high-throughput T2I inference system that selects the right level of approximation for each prompt to maintain quality while meeting throughput targets on a fixed-size cluster. Argus intelligently switches between different approximation strategies to satisfy both throughput and quality requirements. Overall, Argus achieves 10x fewer latency service-level objective (SLO) violations, 10% higher average quality, and 40% higher throughput compared to baselines on two real-world workload traces.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。