让复杂视频查询又快又准，像查数据库一样操作视频数据。
LazyVLM: Neuro-Symbolic Approach to Video Analytics
- 将多帧视频查询拆解为细粒度操作，用关系查询和向量搜索替代大模型全程计算
- 在多个数据集上实现高效响应，处理长视频时比端到端模型快数倍
- 适合需要灵活查询海量视频的开发者或研究人员，无需写代码
当前视频分析方法在灵活性与效率间面临根本权衡。端到端视觉语言模型（VLMs）常因长上下文处理困难而效率低下，神经符号方法则依赖人工标注和固定规则。本文提出LazyVLM，一种神经符号视频分析系统，提供类似VLM的用户友好的查询接口，同时克服其可扩展性局限。用户可轻松导入视频数据，并通过半结构化文本界面指定复杂的多帧视频查询。为解决VLMs的可扩展性问题，LazyVLM将多帧视频查询分解为细粒度操作，将大部分处理任务交由高效的关联查询执行与向量相似性搜索完成。实验表明，LazyVLM能以高鲁棒性、高效率、易用性实现对开放域视频数据的大规模查询。
原文摘要 · Abstract (English)
Current video analytics approaches face a fundamental trade-off between flexibility and efficiency. End-to-end Vision Language Models (VLMs) often struggle with long-context processing and incur high computational costs, while neural-symbolic methods depend heavily on manual labeling and rigid rule design. In this paper, we introduce LazyVLM, a neuro-symbolic video analytics system that provides a user-friendly query interface similar to VLMs, while addressing their scalability limitation. LazyVLM enables users to effortlessly drop in video data and specify complex multi-frame video queries using a semi-structured text interface for video analytics. To address the scalability limitations of VLMs, LazyVLM decomposes multi-frame video queries into fine-grained operations and offloads the bulk of the processing to efficient relational query execution and vector similarity search. We demonstrate that LazyVLM provides a robust, efficient, and user-friendly solution for querying open-domain video data at scale.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。