arXiv:2606.01604cs.CV2026-06TPAMI

用偏微分方程建模点云视频时空变化,提升动作理解能力

Paving the Way for Point Cloud Video Representation Learning Using A PDE Model

论文配图:Paving the Way for Point Cloud Video Representation Learning Using A PDE Model
图 1 · 摘自论文原文
  • 将点云视频的时序变化建模为可解偏微分方程
  • 在S3DIS和ScanNet上达到领先自监督性能,优于基线10%以上
  • 适合作为插件模块嵌入现有模型,计算开销极小

研究点云视频中的时空相关性,特别是空间点随时间的变化规律,对理解动态场景至关重要。传统基于光流的方法因点云数据无序排列而难以捕捉此类相关性。为此,我们提出一种新方法,将时空相关性学习建模为可解的偏微分方程(PDE)。受流体分析启发,构建简化PDE,并通过时序嵌入与空间嵌入间的对比学习结构引导和优化求解过程。所提方法MotionPDE作为通用增强模块,可无缝集成至现有骨干网络,仅引入极少计算开销与参数量。借助对比学习机制,进一步挖掘其自监督潜力,在S3DIS和ScanNet数据集上取得优异结果,验证了其在点云视频理解中的有效性与适应性。代码与训练权重已开源。

原文摘要 · Abstract (English)

Investigating spatial-temporal correlations, specifically how spatial points vary over time, is crucial for understanding point cloud videos. Traditional methods, particularly flow-based techniques, struggle with these correlations due to the unordered spatial arrangement of sequential point cloud data. To address this challenge, we propose a novel approach that regularizes spatial-temporal correlation learning by formulating the problem as a solvable Partial Differential Equation (PDE). While PDEs have long been effective in the physical domain, their application to novel sequential data like point cloud video remains underexplored. Inspired by fluid analysis, we construct a simplified PDE, and the process of solving PDE is guided and refined by a contrastive learning structure between the temporal embeddings and the spatial embeddings. With this extra supervision, our method, named MotionPDE, serves as an effective, plug-and-play enhancement module for existing backbone models, adding minimal computational overhead and parameters. Capitalizing on the contrastive learning process, we delve deeper into the self-supervised capabilities of MotionPDE, yielding promising results that underscore its utility and adaptability in point cloud video data interpretation. The code repo with trained checkpoints will be available at https://github.com/zhh6425/motionpde.git for facilitating future research.

点云视频偏微分方程自监督学习动作理解

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。