arXiv:2409.12190cs.ROcs.CV2024-09被引 8

将捆绑调整引入PyTorch eager模式,实现高效端到端优化。

Bundle Adjustment in the Eager Mode

  • 基于PyTorch构建,支持自动微分与图计算
  • 在多个基准上比GTSAM等工具快18.5至23倍
  • 适合需融合深度学习的机器人与视觉系统开发者

捆绑调整(BA)是同时定位与地图构建(SLAM)、增强现实(AR)和摄影测量等机器人应用中的关键技术。它通过优化相机位姿和三维特征点等参数以对齐观测数据。随着深度学习在感知系统中的重要性提升,亟需将BA与PyTorch等现代深度学习框架集成,以增强可靠性与性能。然而,广泛使用的基于C++的BA库如GTSAM、g²o和Ceres Solver缺乏对PyTorch的原生支持,限制了灵活性、可调试性和实现效率。为此,我们提出一个与PyTorch无缝集成的急切模式BA库,具备稀疏感知的自动微分设计和针对二阶优化的GPU加速稀疏运算。该方法在GPU上表现出显著的运行效率,在所有基准测试中相较GTSAM、g²o和Ceres分别实现了平均18.5×、22×和23×的加速。

原文摘要 · Abstract (English)

Bundle adjustment (BA) is a critical technique in various robotic applications such as simultaneous localization and mapping (SLAM), augmented reality (AR), and photogrammetry. BA optimizes parameters such as camera poses and 3D landmarks to align them with observations. With the growing importance of deep learning in perception systems, there is an increasing need to integrate BA with deep learning frameworks for enhanced reliability and performance. However, widely-used C++-based BA libraries, such as GTSAM, g$^2$o, and Ceres Solver, lack native integration with modern deep learning libraries like PyTorch. This limitation affects their flexibility, ease of debugging, and overall implementation efficiency. To address this gap, we introduce an eager-mode BA library seamlessly integrated with PyTorch with high efficiency. Our approach includes a sparsity-aware auto-differentiation design and GPU-accelerated sparse operations designed for 2nd-order optimization. Our eager-mode BA on GPU demonstrates substantial runtime efficiency, achieving an average speedup of 18.5$\times$, 22$\times$, and 23$\times$ across all benchmarks compared to GTSAM, g$^2$o, and Ceres, respectively.

Bundle AdjustmentPyTorch优化机器人

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。