用动态步长显式建模几何结构,让状态空间模型高效处理稀疏点云和事件数据。
STREAM: A Universal State-Space Model for Sparse Geometric Data
- 将坐标相对差融入状态空间模型步长,显式编码几何结构
- 在11类DVS128手势数据上首次实现100%测试准确率
- 支持从点云到事件视觉的通用建模,适合稀疏几何数据任务
处理稀疏无结构几何数据(如点云或事件视觉)是机器视觉中的关键挑战。近期,序列模型如Transformer和状态空间模型被引入几何数据领域,但需特殊预处理以生成序列视图。先前方法采用固定或学习的步长迭代几何数据,隐式依赖模型推断几何结构。本文提出将几何结构显式编码至状态空间模型参数中。状态空间模型基于一维变量(如时间或空间坐标)的线性动态,我们利用该变量注入坐标的相对差,调节状态空间模型的步长。由此生成的几何操作可在O(N)步内计算所有N个点之间的相互作用。模型采用改进的CUDA核实现的Mamba选择性状态空间模型,高效适配现代硬件。所提序列模型STREAM在点云分类、事件视觉和音频分类等多类基准上表现优异。其在ModelNet40和ScanObjectNN点云分析数据集上优于PointMamba基线;首次在DVS128手势数据集的全部11类上达到100%测试准确率。
原文摘要 · Abstract (English)
Handling sparse and unstructured geometric data, such as point clouds or event-based vision, is a pressing challenge in the field of machine vision. Recently, sequence models such as Transformers and state-space models entered the domain of geometric data. These methods require specialized preprocessing to create a sequential view of a set of points. Furthermore, prior works involving sequence models iterate geometric data with either uniform or learned step sizes, implicitly relying on the model to infer the underlying geometric structure. In this work, we propose to encode geometric structure explicitly into the parameterization of a state-space model. State-space models are based on linear dynamics governed by a one-dimensional variable such as time or a spatial coordinate. We exploit this dynamic variable to inject relative differences of coordinates into the step size of the state-space model. The resulting geometric operation computes interactions between all pairs of N points in O(N) steps. Our model deploys the Mamba selective state-space model with a modified CUDA kernel to efficiently map sparse geometric data to modern hardware. The resulting sequence model, which we call STREAM, achieves competitive results on a range of benchmarks from point-cloud classification to event-based vision and audio classification. STREAM demonstrates a powerful inductive bias for sparse geometric data by improving the PointMamba baseline when trained from scratch on the ModelNet40 and ScanObjectNN point cloud analysis datasets. It further achieves, for the first time, 100% test accuracy on all 11 classes of the DVS128 Gestures dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。