让多个任务共享同一个大模型，显著降低延迟并提升部署密度。
FMplex: Model Virtualization for Serving Extensible Foundation Models
- 将大模型抽象为虚拟实例，实现多任务共享同一物理模型
- 在7个大模型、92个任务上降低80%延迟，支持6倍更多任务并发
- 适合需要高效部署多种定制化任务的工业级AI系统
基础模型（FMs）正被广泛应用于语言、视觉、时序及多模态等下游任务。然而现有服务系统将每个定制任务独立部署为模型实例，导致重型基础模型重复加载，浪费加速器内存，并错失批处理和加载成本分摊的机会。本文提出FMplex，一个将基础模型作为虚拟化底座的服务系统。通过为每个任务提供虚拟基础模型（vFM），实现逻辑隔离的同时共享物理模型。该设计支持任务特定扩展、独立生命周期与任务级隔离。此外，我们提出一种感知批处理的公平队列调度器，结合任务间与任务内批处理，实现加权共享。我们构建了涵盖任务构建、共享感知部署和运行时执行的完整服务栈。在7个基础模型（16种变体）和92个下游任务上，FMplex相比空间分区降低80%延迟，相比最佳努力共置降低33.3%延迟，集群规模下可承载最多6倍的任务数量。
原文摘要 · Abstract (English)
Foundation models (FMs) are increasingly used as backbones for downstream tasks across language, vision, time-series, and multimodal applications. Yet existing model-serving systems deploy each customized task as an independent model instance, thereby replicating heavyweight backbones, wasting accelerator memory, and losing opportunities to amortize batching and loading costs. This paper presents FMplex, a serving system that treats FM backbones as a virtualization substrate for deployment sharing. FMplex presents each task with a virtual foundation model (vFM), a logically private FM instance backed by a shared physical FM. This abstraction lets independently customized tasks share a backbone while preserving task-specific extensions, independent lifecycles, and task-level isolation. In addition, we propose a batch-aware fair-queueing scheduler that combines weighted task-level sharing with inter- and intra-task batching across colocated tasks. We implement a FMplex-based serving stack spanning task construction, sharing-aware deployment, and runtime execution. Across 7 FM backbones (16 variants) and 92 downstream tasks, FMplex reduces latency by up to 80% over spatial partitioning and 33.3% over best-effort co-location, while hosting up to 6x more tasks at cluster scale.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。