arXiv:2507.14301cs.IRcs.CV2025-07

LOVO系统实现高效复杂物体查询,支持大规模视频数据低延迟检索。

LOVO: Efficient Complex Object Query in Large-Scale Video Datasets

  • 预提取关键帧特征构建紧凑嵌入索引,支持任意物体查询。
  • 查询时通过近似最近邻搜索,实现85倍降速与高准确率。
  • 适合需要快速响应的智能监控、交通管理等场景。

摄像头的广泛部署导致视频数据呈指数级增长,为交通管理与犯罪监控等应用带来机遇。然而,从大规模视频数据集中查询特定物体面临三大挑战:处理海量持续增长的数据、支持复杂查询需求、保证低延迟执行。现有方法在应对未见物体类别时适应性差或查询延迟高。本文提出LOVO系统,用于高效处理大规模视频数据中的复杂物体查询。该系统对用户查询无感,利用预训练视觉编码器一次性提取关键帧特征,生成紧凑视觉嵌入以构建高效索引。这些视觉嵌入及对应边界框被组织在向量数据库的倒排多索引结构中,支持任意物体查询。查询阶段,将物体查询转换为查询嵌入,并在视觉嵌入上进行快速近似最近邻搜索。最后,通过跨模态重排序融合视觉与文本特征以优化结果。在真实世界视频数据集上的评估表明,相比现有方法,LOVO在处理复杂查询时达到近优精度,搜索延迟降低高达85倍,同时显著降低索引构建成本。该系统重新定义了视频分析中物体查询的前沿标准,提出了新颖、可扩展且高效的动态环境查询方案。

原文摘要 · Abstract (English)

The widespread deployment of cameras has led to an exponential increase in video data, creating vast opportunities for applications such as traffic management and crime surveillance. However, querying specific objects from large-scale video datasets presents challenges, including (1) processing massive and continuously growing data volumes, (2) supporting complex query requirements, and (3) ensuring low-latency execution. Existing video analysis methods struggle with either limited adaptability to unseen object classes or suffer from high query latency. In this paper, we present LOVO, a novel system designed to efficiently handle comp$\underline{L}$ex $\underline{O}$bject queries in large-scale $\underline{V}$ide$\underline{O}$ datasets. Agnostic to user queries, LOVO performs one-time feature extraction using pre-trained visual encoders, generating compact visual embeddings for key frames to build an efficient index. These visual embeddings, along with associated bounding boxes, are organized in an inverted multi-index structure within a vector database, which supports queries for any objects. During the query phase, LOVO transforms object queries to query embeddings and conducts fast approximate nearest-neighbor searches on the visual embeddings. Finally, a cross-modal rerank is performed to refine the results by fusing visual features with detailed textual features. Evaluation on real-world video datasets demonstrates that LOVO outperforms existing methods in handling complex queries, with near-optimal query accuracy and up to 85x lower search latency, while significantly reducing index construction costs. This system redefines the state-of-the-art object query approaches in video analysis, setting a new benchmark for complex object queries with a novel, scalable, and efficient approach that excels in dynamic environments.

视频查询向量索引低延迟跨模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。