提出弹性多模态并行机制,显著降低多模态大模型推理延迟。
ElasticMM: Efficient Multimodal LLMs Serving with Elastic Multimodal Parallelism
- 按模态分组动态分配资源,实现请求级灵活调度。
- 推理阶段可调并行度,最高提升4.5倍吞吐量。
- 适合高并发多模态服务场景,如图像问答与视频理解。
多模态大语言模型(MLLM)通过引入特征提取器和投影模块扩展了传统LLM对图像、视频和音频的处理能力。然而,这些额外组件结合复杂的推理流程与异构负载,带来了显著的推理开销。当前紧耦合的服务架构难以区分混合请求类型或根据推理阶段调整并行策略,导致首次生成时间(TTFT)增加且资源利用率低下。为此,本文提出弹性多模态并行(EMP)新范式,能动态适应不同请求类型与推理阶段的资源异质性。基于EMP,我们构建ElasticMM系统:(1) 通过模态感知负载均衡器将请求分离为独立模态组并动态分配资源;(2) 解耦推理阶段,支持弹性分区调度以自适应调整并行度与扩展规模;(3) 通过统一的多模态前缀缓存与非阻塞编码提升推理效率。在多个真实世界数据集上的实验表明,ElasticMM优于现有最优服务系统,在满足服务等级目标(SLO)前提下,将TTFT降低最多4.2倍,吞吐量提升3.2–4.5倍。
原文摘要 · Abstract (English)
Multimodal large language models (MLLMs) extend LLMs to handle images, videos, and audio by incorporating feature extractors and projection modules. However, these additional components -- combined with complex inference pipelines and heterogeneous workloads -- introduce significant inference overhead. Therefore, efficiently serving MLLMs remains a major challenge. Current tightly coupled serving architectures struggle to distinguish between mixed request types or adapt parallelism strategies to different inference stages, leading to increased time-to-first-token (TTFT) latency and poor resource utilization. To address this, we introduce Elastic Multimodal Parallelism (EMP), a new serving paradigm that elastically adapts to resource heterogeneity across request types and inference stages. Building upon EMP, we develop ElasticMM, an MLLM serving system that (1) separates requests into independent modality groups with dynamic resource allocation via a modality-aware load balancer; (2) decouples inference stages and enables parallelism adjustment and adaptive scaling via elastic partition scheduling; and (3) improves inference efficiency through unified multimodal prefix caching and non-blocking encoding. Experiments on diverse real-world datasets show that ElasticMM outperforms state-of-the-art (SOTA) serving systems, reducing TTFT by up to 4.2x and achieving 3.2-4.5x higher throughput while meeting service-level objectives (SLOs).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。