StreamWise实现实时大规模多模态生成，低延迟高效部署。
StreamWise: Serving Multi-Modal Generation in Real-Time at Scale
- 动态调整分辨率与资源分配，按需调度多模态模型。
- 10分钟播客视频生成耗时1.4小时，成本低于25美元。
- 支持亚秒级启动延迟，适合实时流媒体应用。
多模态生成模型正推动故事创作与自动化媒体合成等新应用发展。当前多数工作负载以批处理方式生成简单输出（如从提示生成图像），即使基础结果也需数秒。在严格延迟与资源约束下，实现大规模实时多模态工作流服务成本高、复杂度大，需协调语言、音频、图像和视频等不同模型，各自具有独特资源需求。本文以实时播客视频生成为例，集成大语言模型（LLM）、文本转语音（TTS）与音视频生成模型，提出自适应模块化服务系统StreamWise，动态管理质量（如分辨率、锐度）、模型/内容并行性及资源感知调度。利用异构硬件提升响应速度与效率。例如，系统可降低视频分辨率以释放资源用于早期场景。量化了延迟、成本与质量间的权衡：最廉价配置在A100 GPU上生成10分钟播客视频耗时1.4小时（慢于实时8.4倍），成本低于25美元；而StreamWise可在45美元内实现高质量实时流，启动延迟低于1秒。
原文摘要 · Abstract (English)
Advances in multi-modal generative models are enabling new applications, from storytelling to automated media synthesis. Most current workloads generate simple outputs (e.g., image generation from a prompt) in batch mode, often requiring several seconds even for basic results. Serving real-time multi-modal workflows at scale is costly and complex, requiring efficient coordination of diverse models (each with unique resource needs) across language, audio, image, and video, all under strict latency and resource constraints. We tackle these challenges through the lens of real-time podcast video generation, integrating LLMs, text-to-speech, and video-audio generation. To meet tight SLOs, we design an adaptive, modular serving system, StreamWise, that dynamically manages quality (e.g., resolution, sharpness), model/content parallelism, and resource-aware scheduling. We leverage heterogeneous hardware to maximize responsiveness and efficiency. For example, the system can lower video resolution and allocate more resources to early scenes. We quantify the trade-offs between latency, cost, and quality. The cheapest setup generates a 10-minute podcast video on A100 GPUs in 1.4 hours (8.4x slower than the real-time) for less than \$25. StreamWise enables high-quality real-time streaming with a sub-second startup delay under $45.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。