用点级令牌优化协同感知,提升3D物体识别精度。
Beyond BEV: Optimizing Point-Level Tokens for Collaborative Perception
- 以点级令牌替代传统鸟瞰图表示,保留精细3D结构信息。
- 在真实与模拟数据集上显著优于现有方法,通信开销更低。
- 适合自动驾驶多智能体协同感知场景,尤其关注定位精度的系统设计者。
协同感知使智能体通过交换中间特征来增强感知能力。现有方法通常将中间特征组织为二维鸟瞰图(BEV)表示,但会丢失对准确物体识别与定位至关重要的细粒度3D结构线索。为此,本文首次引入点级令牌作为协同感知的中间表示。然而,点云数据本质上无序、海量且对位置敏感,难以生成紧凑且对齐的点级令牌序列以保留详细结构信息。因此,我们提出CoPLOT,一种新型协同感知框架,采用点级优化令牌(Point-Level Optimized Tokens)。该框架包含原生点处理流程:令牌重排、序列建模与多智能体空间对齐。语义感知的令牌重排模块利用场景级与令牌级语义信息生成自适应的一维重排;频域增强的状态空间模型在空间与谱域中捕捉长程序列依赖,提升前景令牌与背景杂波的区分度;邻近到自身对齐模块通过闭环过程,结合全局代理级修正与局部令牌级精炼,减轻定位噪声。在模拟与真实世界数据集上的大量实验表明,CoPLOT优于最先进模型,且通信与计算开销更低。代码将发布于 https://github.com/CheeryLeeyy/CoPLOT。
原文摘要 · Abstract (English)
Collaborative perception allows agents to enhance their perceptual capabilities by exchanging intermediate features. Existing methods typically organize these intermediate features as 2D bird's-eye-view (BEV) representations, which discard critical fine-grained 3D structural cues essential for accurate object recognition and localization. To this end, we first introduce point-level tokens as intermediate representations for collaborative perception. However, point-cloud data are inherently unordered, massive, and position-sensitive, making it challenging to produce compact and aligned point-level token sequences that preserve detailed structural information. Therefore, we present CoPLOT, a novel Collaborative perception framework that utilizes Point-Level Optimized Tokens. It incorporates a point-native processing pipeline, including token reordering, sequence modeling, and multi-agent spatial alignment. A semantic-aware token reordering module generates adaptive 1D reorderings by leveraging scene-level and token-level semantic information. A frequency-enhanced state space model captures long-range sequence dependencies across both spatial and spectral domains, improving the differentiation between foreground tokens and background clutter. Lastly, a neighbor-to-ego alignment module applies a closed-loop process, combining global agent-level correction with local token-level refinement to mitigate localization noise. Extensive experiments on both simulated and real-world datasets show that CoPLOT outperforms state-of-the-art models, with even lower communication and computation overhead. Code will be available at https://github.com/CheeryLeeyy/CoPLOT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。