QuickGrasp让视频语言查询更快更准,本地优先+按需调用边缘算力。
QuickGrasp: Responsive Video-Language Querying Service via Accelerated Tokenization and Edge-Augmented Inference
- 本地先跑+按需调用边缘算力,减少延迟。
- 响应速度提升12.8倍,准确率媲美大模型。
- 适合对实时性要求高的视频理解场景。
视频-语言模型(VLMs)正在重塑视频查询服务,为复杂感知与推理任务提供统一解决方案。然而,由于资源消耗高,大型VLM在真实系统中部署仍具挑战性,远程部署常导致不可接受的响应延迟。尽管小型本地模型响应快,但准确率明显下降。为此,我们提出QuickGrasp——一种面向服务质量(QoS)的响应式系统,通过本地优先架构与按需边缘增强来弥合这一差距。基于VLM的高度模块化设计,QuickGrasp共享视觉表征以避免冗余计算。为最大化系统效率,引入三项关键设计:加速视频分词、查询自适应边缘增强和延迟感知、准确率保持的视觉分词密度配置。我们实现了一个QuickGrasp原型,并在多个视频理解基准上评估。结果表明,QuickGrasp在保持大型VLM准确率的同时,将响应延迟降低高达12.8倍。该工作推动了开放世界理解下高效响应视频查询服务的发展。
原文摘要 · Abstract (English)
Video-language models (VLMs) are reshaping video querying services, bringing unified solutions to complex perception and reasoning tasks. However, deploying large VLMs in real-world systems remains challenging due to their high resource demands, and remote-based deployment often results in unacceptable response delays. Although small, locally deployable VLMs offer faster responses, they unavoidably fall short in accuracy. To reconcile this trade-off, we propose QuickGrasp, a responsive, quality of service (QoS)-aware system that bridges this gap through a local-first architecture with on-demand edge augmentation. Built upon the highly modular architecture of VLMs, QuickGrasp shares the vision representation across model variants to avoid redundant computation. To maximize system-wide efficiency, QuickGrasp introduces three key designs: accelerated video tokenization, query-adaptive edge augmentation, and delay-aware, accuracy-preserving vision token density configuration. We implement a prototype of QuickGrasp and evaluate it across multiple video understanding benchmarks. The results show that QuickGrasp matches the accuracy of large VLMs while achieving up to a 12.8x reduction in response delay. QuickGrasp represents a key advancement toward building responsive video querying services for open-world understanding that fully leverage the capabilities of VLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。