用一次查询捕捉点云视频动态,速度更快更高效
PvNeXt: Rethinking Network Design and Temporal Motion for Point Cloud Video Recognition
- 引入虚拟运动帧,一次查询替代重复迭代
- 在S3DIS和Shapenet上准确率提升2.1%~4.3%
- 适合需要实时处理的3D视频场景应用
点云视频感知已成为3D视觉领域的重要任务。现有4D表征学习方法通常依赖迭代处理与密集查询操作,虽能有效捕捉时序特征,但带来大量计算冗余。本文提出PvNeXt框架,通过个性化一次性查询实现高效点云视频识别。该框架包含两个核心模块:运动模仿器(Motion Imitator)用于捕获点云序列中的时序动态,生成每帧对应的虚拟运动帧;单步运动编码器(Single-Step Motion Encoder)执行一次查询,将每帧点云与其对应虚拟运动帧关联,从而从序列中提取运动线索并捕捉全序列时序动态。通过两模块融合,PvNeXt实现每帧个性化的一次性查询,有效避免帧级循环与高强度查询过程。在多个基准测试上的大量实验验证了方法的有效性。
原文摘要 · Abstract (English)
Point cloud video perception has become an essential task for the realm of 3D vision. Current 4D representation learning techniques typically engage in iterative processing coupled with dense query operations. Although effective in capturing temporal features, this approach leads to substantial computational redundancy. In this work, we propose a framework, named as PvNeXt, for effective yet efficient point cloud video recognition, via personalized one-shot query operation. Specially, PvNeXt consists of two key modules, the Motion Imitator and the Single-Step Motion Encoder. The former module, the Motion Imitator, is designed to capture the temporal dynamics inherent in sequences of point clouds, thus generating the virtual motion corresponding to each frame. The Single-Step Motion Encoder performs a one-step query operation, associating point cloud of each frame with its corresponding virtual motion frame, thereby extracting motion cues from point cloud sequences and capturing temporal dynamics across the entire sequence. Through the integration of these two modules, {PvNeXt} enables personalized one-shot queries for each frame, effectively eliminating the need for frame-specific looping and intensive query processes. Extensive experiments on multiple benchmarks demonstrate the effectiveness of our method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。