剖析多模态生成模型推理瓶颈，提出高效加速方案
Characterizing and Efficiently Accelerating Multimodal Generation Model Inference
- 分析真实系统中多模态生成模型的延迟瓶颈
- 发现自回归生成与线性计算是主要延迟来源，优化后提速3.88倍
- 适合关注AI推理优化、系统设计的研究者与工程师
生成式人工智能技术正重塑计算产业，其应用已扩展至多个领域，并带来新的系统设计与优化机遇。该技术具备跨模态理解与生成能力，但当前高能力伴随显著资源消耗。为实现全球数十亿用户规模的可持续扩展，推理必须快速高效。本文通过在真实系统上表征一类新兴多模态生成模型，揭示关键系统设计与优化机会：自回归标记生成是主要延迟瓶颈，通常由GPU空闲时间主导；除生成式AI中内存密集的注意力机制外，基于Transformer的前馈网络中的线性操作也构成显著推理延迟。实验表明，从应用到系统软件再到硬件的先进优化手段，可将基线性能提升3.88倍。
原文摘要 · Abstract (English)
Generative artificial intelligence (AI) technology is revolutionizing the computing industry. Not only its applications have broadened to various sectors but also poses new system design and optimization opportunities. The technology is capable of understanding and responding in multiple modalities. However, the advanced capability currently comes with significant system resource demands. To sustainably scale generative AI capabilities to billions of users in the world, inference must be fast and efficient. This paper pinpoints key system design and optimization opportunities by characterizing a family of emerging multi-modal generation models on real systems. Auto-regressive token generation is a critical latency performance bottleneck, typically dominated by GPU idle time. In addition to memory-intensive attention across the generative AI models, linear operations constitute significant inference latency due to the feed forward networks in Transformer-based models. We demonstrate that state-of-the-art optimization levers, spanning from applications to system software and hardware, set a 3.88x better baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。