实时处理动态场景4D语义分割,双线程提升精度与速度
4DSegStreamer: Streaming 4D Panoptic Segmentation via Dual Threads
- 采用预测与推理双线程架构,协同处理时序信息
- 在HOI4D、SemanticKITTI等数据集上实现高帧率下精准动态物体预测
- 适合自动驾驶、密集人群疏散等实时感知场景
在高动态环境(如密集人群疏散、复杂场景自动驾驶)中,流式4D语义分割对实时、细粒度感知至关重要。本文提出4DSegStreamer,一种基于双线程系统的新型框架,可高效处理连续帧流。该框架通用性强,能无缝集成至现有3D/4D分割方法中,实现实时性能。系统包含预测线程与推理线程:预测线程利用历史运动与几何信息提取特征并预测未来动态;推理线程通过与最新记忆对齐,补偿自车运动及动态物体位移,确保新帧及时输出。我们在室内HOI4D与室外SemanticKITTI、nuScenes数据集上进行评估,实验表明该方法在复杂场景中对动态物体的预测具有更高准确性与鲁棒性,尤其在高帧率条件下表现优异。
原文摘要 · Abstract (English)
4D panoptic segmentation in a streaming setting is critical for highly dynamic environments, such as evacuating dense crowds and autonomous driving in complex scenarios, where real-time, fine-grained perception within a constrained time budget is essential. In this paper, we introduce 4DSegStreamer, a novel framework that employs a Dual-Thread System to efficiently process streaming frames. The framework is general and can be seamlessly integrated into existing 3D and 4D segmentation methods to enable real-time capability. It also demonstrates superior robustness compared to existing streaming perception approaches, particularly under high FPS conditions. The system consists of a predictive thread and an inference thread. The predictive thread leverages historical motion and geometric information to extract features and forecast future dynamics. The inference thread ensures timely prediction for incoming frames by aligning with the latest memory and compensating for ego-motion and dynamic object movements. We evaluate 4DSegStreamer on the indoor HOI4D dataset and the outdoor SemanticKITTI and nuScenes datasets. Comprehensive experiments demonstrate the effectiveness of our approach, particularly in accurately predicting dynamic objects in complex scenes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。