在边缘设备上用双层语义画布实现低延迟视觉感知
Motion-Compensated Latent Semantic Canvases for Visual Situational Awareness on Edge
- 用静态与动态双层潜在画布存储语义信息，通过运动补偿保持坐标一致
- 相比逐帧分割，分割调用减少30倍以上，端到端处理时间降低20倍以上
- 适合资源受限的实时视觉场景理解，如自动驾驶边缘计算
我们提出运动补偿潜在语义画布（MCLSC），用于资源受限边缘设备上的视觉情境感知。核心思想是在基于视频流稳定化的基线坐标系中，维护两个潜在画布——一个缓慢积累的静态层和一个快速更新的动态层——来持久化语义元数据。昂贵的全景分割（Mask2Former）异步运行并受运动门控：仅当运动表明有新信息时才触发推理，而稳定化/运动补偿机制确保了潜在语义记忆的一致坐标系统。在预录的480p视频片段上，原型系统将分割调用次数减少超过30倍，端到端平均处理时间降低超过20倍，同时保持静态与动态语义叠加的一致性。
原文摘要 · Abstract (English)
We propose Motion-Compensated Latent Semantic Canvases (MCLSC) for visual situational awareness on resource-constrained edge devices. The core idea is to maintain persistent semantic metadata in two latent canvases - a slowly accumulating static layer and a rapidly updating dynamic layer - defined in a baseline coordinate frame stabilized from the video stream. Expensive panoptic segmentation (Mask2Former) runs asynchronously and is motion-gated: inference is triggered only when motion indicates new information, while stabilization/motion compensation preserves a consistent coordinate system for latent semantic memory. On prerecorded 480p clips, our prototype reduces segmentation calls by >30x and lowers mean end-to-end processing time by >20x compared to naive per-frame segmentation, while maintaining coherent static/dynamic semantic overlays.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。