arXiv:2508.14237cs.NIcs.CV2025-08中稿 · INFOCOM'23被引 8

为360度视频设计边缘智能分析系统,低延迟高精度提取关键信息。

OmniSense: Towards Edge-Assisted Online Analytics for 360-Degree Videos

  • 通过轻量级球面兴趣区预测,动态裁剪冗余视频内容。
  • 在真实场景中实现19.8%~114.6%的准确率提升,延迟相近。
  • 适合实时沉浸式应用如VR/AR、智慧城市监控等场景。

随着全向摄像头硬件成本降低及扩展现实应用普及,越来越多360°视频被采集。为充分释放其潜力,需通过先进视频分析提取无盲点的可行动洞察与情境知识。本文提出OmniSense,一种新型边缘协同在线沉浸式视频分析框架。该框架在应对360°视频分析中的计算与网络资源挑战时,兼顾低延迟与高精度。基于对360°视频的测量洞察,OmniSense引入轻量级球面兴趣区(SRoI)预测算法,剔除冗余帧信息;结合视频内容与网络动态,智能缩放视觉模型以优化资源利用。我们在通用设备上实现了OmniSense原型,并在多种真实采集的360°视频上进行了评估。结果表明,相比资源无感知基线,其准确率提升19.8%~114.6%,端到端延迟相当;同时获得2.0×~2.4×的加速比,且准确率与基线最高水平持平。

原文摘要 · Abstract (English)

With the reduced hardware costs of omnidirectional cameras and the proliferation of various extended reality applications, more and more $360^\circ$ videos are being captured. To fully unleash their potential, advanced video analytics is expected to extract actionable insights and situational knowledge without blind spots from the videos. In this paper, we present OmniSense, a novel edge-assisted framework for online immersive video analytics. OmniSense achieves both low latency and high accuracy, combating the significant computation and network resource challenges of analyzing $360^\circ$ videos. Motivated by our measurement insights into $360^\circ$ videos, OmniSense introduces a lightweight spherical region of interest (SRoI) prediction algorithm to prune redundant information in $360^\circ$ frames. Incorporating the video content and network dynamics, it then smartly scales vision models to analyze the predicted SRoIs with optimized resource utilization. We implement a prototype of OmniSense with commodity devices and evaluate it on diverse real-world collected $360^\circ$ videos. Extensive evaluation results show that compared to resource-agnostic baselines, it improves the accuracy by $19.8\%$ -- $114.6\%$ with similar end-to-end latencies. Meanwhile, it hits $2.0\times$ -- $2.4\times$ speedups while keeping the accuracy on par with the highest accuracy of baselines.

360视频边缘计算智能分析VR/AR

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。