通过模块复用提升移动端视频语言模型运行效率
Atom: Efficient On-Device Video-Language Pipelines Through Modular Reuse
- 将大模型拆分为可复用的视觉编码器和语言解码器模块
- 在手机上实现27%~33%更快的端到端延迟，性能损失极小
- 适合需要高效部署视频理解应用的移动端开发者
视频语言模型近年推动了视频检索、字幕生成与组合等应用的发展。然而，在移动设备上高效执行多阶段流水线仍面临模型重复加载和执行碎片化的问题。我们提出Atom，一种面向设备端的系统，通过重构视频语言流水线实现快速高效执行。Atom将百亿参数模型分解为可复用的模块（如视觉编码器、语言解码器），并在字幕生成、推理、索引等子任务间共享这些模块。该复用设计避免了重复加载，支持并行执行，显著降低端到端延迟，且不牺牲性能。在主流智能手机上，Atom相比非复用基线提速27%~33%，检索召回率下降不超过2.3（Recall@1），字幕生成得分下降不超过1.5（CIDEr）。Atom为边缘设备上的视频语言理解提供了实用且可扩展的解决方案。
原文摘要 · Abstract (English)
Recent advances in video-language models have enabled powerful applications like video retrieval, captioning, and assembly. However, executing such multi-stage pipelines efficiently on mobile devices remains challenging due to redundant model loads and fragmented execution. We introduce Atom, an on-device system that restructures video-language pipelines for fast and efficient execution. Atom decomposes a billion-parameter model into reusable modules, such as the visual encoder and language decoder, and reuses them across subtasks like captioning, reasoning, and indexing. This reuse-centric design eliminates repeated model loading and enables parallel execution, reducing end-to-end latency without sacrificing performance. On commodity smartphones, Atom achieves 27--33% faster execution compared to non-reuse baselines, with only marginal performance drop ($\leq$ 2.3 Recall@1 in retrieval, $\leq$ 1.5 CIDEr in captioning). These results position Atom as a practical, scalable approach for efficient video-language understanding on edge devices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。