用图像引导3D点云补全,提升机器人感知精度
HGACNet: Hierarchical Graph Attention Network for Cross-Modal Point Cloud Completion
- 分层图注意力网络提取点云几何特征
- 跨模态融合模块实现图像与点云精准对齐
- 适合机器人抓取、避障等实际场景应用
点云补全是机器人感知、物体重建及后续任务(如抓取规划、避障和操作)的关键。然而,由于自遮挡和传感器限制导致的几何不完整,会显著影响下游推理与交互。为此,我们提出HGACNet,一种通过分层编码三维几何特征并融合单视角RGB图像的语义先验来重建完整点云的新框架。核心在于分层图注意力(HGA)编码器,通过基于图注意力的下采样自适应选择关键局部点,并逐级优化几何特征,以更好捕捉结构连续性与空间关系。为增强跨模态交互,我们设计了多尺度跨模态融合(MSCF)模块,实现层级几何特征与结构化视觉表征间的注意力对齐,提供细粒度语义指导。此外,引入对比损失(C-Loss)显式对齐跨模态特征分布,在模态差异下提升补全保真度。在ShapeNet-ViPC和YCB-Complete数据集上的大量实验验证了HGACNet的有效性,达到当前最优性能,并展现出在真实机器人操作任务中的强适用性。
原文摘要 · Abstract (English)
Point cloud completion is essential for robotic perception, object reconstruction and supporting downstream tasks like grasp planning, obstacle avoidance, and manipulation. However, incomplete geometry caused by self-occlusion and sensor limitations can significantly degrade downstream reasoning and interaction. To address these challenges, we propose HGACNet, a novel framework that reconstructs complete point clouds of individual objects by hierarchically encoding 3D geometric features and fusing them with image-guided priors from a single-view RGB image. At the core of our approach, the Hierarchical Graph Attention (HGA) encoder adaptively selects critical local points through graph attention-based downsampling and progressively refines hierarchical geometric features to better capture structural continuity and spatial relationships. To strengthen cross-modal interaction, we further design a Multi-Scale Cross-Modal Fusion (MSCF) module that performs attention-based feature alignment between hierarchical geometric features and structured visual representations, enabling fine-grained semantic guidance for completion. In addition, we proposed the contrastive loss (C-Loss) to explicitly align the feature distributions across modalities, improving completion fidelity under modality discrepancy. Finally, extensive experiments conducted on both the ShapeNet-ViPC benchmark and the YCB-Complete dataset confirm the effectiveness of HGACNet, demonstrating state-of-the-art performance as well as strong applicability in real-world robotic manipulation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。