用曲率感知注意力提升3D场景描述的精度与连贯性
Curvature-Aware Captioning:Leveraging Geodesic Attention for 3D Scene Understanding

- 引入非欧几里得测地线注意力,分别在斜交空间和洛伦兹空间建模局部与全局关系
- 在ScanRefer和Nr3D上实现定位精度与描述丰富度的双重提升,超越现有方法
- 适合做3D视觉理解、机器人导航或增强现实的开发者参考
准确的3D场景描述对机器人导航和增强现实至关重要,但现有密集标注方法在处理稀疏点云时存在显著局限。传统基于欧几里得嵌入空间的方法难以同时保留精细局部几何细节并建模指数增长的全局语义层次,导致定位不准或描述零散。本文提出新型曲率感知命名框架,结合非欧几里得测地线注意力机制,解决定位与上下文建模的冲突。具体而言,斜交空间中的自注意力确保维度一致性并建立长程依赖;洛伦兹空间中的双向测地线交叉注意力建模场景实例间的层次语义关系,实现对象定位的高精度与场景描述的连贯性。理论分析表明,斜交流形与洛伦兹双曲面之间的曲率互补性可化解欧几里得-双曲冲突,通过各向同性优化保障特征稳定性并保留固有层次结构。在ScanRefer和Nr3D基准上的大量实验表明,该方法达到当前最优性能,定位准确率与描述丰富度均有显著提升。
原文摘要 · Abstract (English)
Accurate 3D scene description is fundamental to robotic navigation and augmented reality, yet current dense captioning methods face significant limitations in processing sparse point cloud data. % Existing approaches that apply Euclidean embedding spaces struggle to simultaneously preserve fine-grained local geometric details and model exponentially growing global semantic hierarchies, leading to either inaccurate localization or disjointed, shallow scene descriptions. % In this work, we propose a novel \textbf{\textsc{Curvature-Aware Captioning}} framework, integrating novel non-Euclidean geodesic attention mechanisms, to resolve the localization-contextualization conflict. % Specifically, self-attention within Oblique space enforces dimensional homogeneity while establishing long-range dependencies. Bidirectional geodesic cross-attention within Lorentz space models hierarchical semantic relationships across scene instances, enabling simultaneous precision in object localization and coherence in scene descriptions. % Theoretical analysis confirms that the curvature complementarity between the Oblique manifold and Lorentz hyperboloid resolves the Euclidean-hyperbolic conflict, ensuring feature stability via isotropic optimization while preserving inherent hierarchical relationships. Extensive experiments on ScanRefer and Nr3D benchmarks demonstrate state-of-the-art performance, with significant gains in both localization accuracy and descriptive richness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。