为事件相机设计低延迟可扩展流媒体,保障检测精度同时适配低功耗设备。
Low-Latency Scalable Streaming for Event-Based Vision
- 基于Media Over QUIC的多流优先级调度,动态丢弃非关键数据流
- 5毫秒延迟下检测平均损失仅0.36,50毫秒时降至0.19
- 适合对延迟敏感且资源受限的嵌入式事件视觉应用
近年来,新型事件相机传感器在高速、低功耗视频采集方面崭露头角。与传统帧图像不同,事件相机以微秒级精度异步输出亮度变化超过阈值的事件元组。尽管这些传感器推动了众多新视觉应用,但现有应用常依赖昂贵高功耗的GPU系统,难以部署在事件相机本就优化的低功耗设备上。当前视频流媒体中至关重要的接收端速率自适应,在事件视觉领域仍研究不足。我们在真实事件相机数据集上首次证明,主流目标检测应用对大幅数据丢失具有鲁棒性,且数据损失可集中在时间窗口末尾。为此,我们提出一种基于Media Over QUIC的事件数据可扩展流媒体方法,优先保障检测性能与低延迟。应用服务器可同时接收多个数据流,并按需丢弃部分流以维持延迟上限。在小网络环境下,端到端延迟目标为5毫秒时,检测平均精度(mAP)下降仅为0.36;延迟放宽至50毫秒时,平均下降低至0.19。
原文摘要 · Abstract (English)
Recently, we have witnessed the rise of novel ``event-based'' camera sensors for high-speed, low-power video capture. Rather than recording discrete image frames, these sensors output asynchronous ``event'' tuples with microsecond precision, only when the brightness change of a given pixel exceeds a certain threshold. Although these sensors have enabled compelling new computer vision applications, these applications often require expensive, power-hungry GPU systems, rendering them incompatible for deployment on the low-power devices for which event cameras are optimized. Whereas receiver-driven rate adaptation is a crucial feature of modern video streaming solutions, this topic is underexplored in the realm of event-based vision systems. On a real-world event camera dataset, we first demonstrate that a state-of-the-art object detection application is resilient to dramatic data loss, and that this loss may be weighted towards the end of each temporal window. We then propose a scalable streaming method for event-based data based on Media Over QUIC, prioritizing object detection performance and low latency. The application server can receive complementary event data across several streams simultaneously, and drop streams as needed to maintain a certain latency. With a latency target of 5 ms for end-to-end transmission across a small network, we observe an average reduction in detection mAP as low as 0.36. With a more relaxed latency target of 50 ms, we observe an average mAP reduction as low as 0.19.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。