用视觉和电机数据融合，实时精准预测柔性机器人的变形形状。
Estimating Continuum Robot Shape under External Loading using Spatiotemporal Neural Networks
- 设计时空神经网络，融合图像与历史位移数据
- 加载下误差仅0.22毫米，优于现有方法
- 适合需要高精度形变感知的医疗机器人场景
本文提出一种基于学习的方法，用于精确估计受外部载荷影响的柔性连续体机器人三维形状。所提方法引入一种时空神经网络架构，融合多模态输入——包括当前及历史腱位移数据和RGB图像——生成表示机器人变形构型的点云。网络集成循环神经模块以提取时间特征，编码模块用于空间特征提取，并设有多模态融合模块，将视觉数据的空间特征与历史执行器输入的时间依赖性相结合。通过将贝塞尔曲线拟合到预测点云，实现连续3D形状重建。实验验证表明，该方法在无负载时平均形状估计误差为0.08毫米，在加载时为0.22毫米，显著优于当前最优的TDCR形状感知方法。结果验证了深度学习驱动的时空数据融合在载荷条件下精确形状估计的有效性。
原文摘要 · Abstract (English)
This paper presents a learning-based approach for accurately estimating the 3D shape of flexible continuum robots subjected to external loads. The proposed method introduces a spatiotemporal neural network architecture that fuses multi-modal inputs, including current and historical tendon displacement data and RGB images, to generate point clouds representing the robot's deformed configuration. The network integrates a recurrent neural module for temporal feature extraction, an encoding module for spatial feature extraction, and a multi-modal fusion module to combine spatial features extracted from visual data with temporal dependencies from historical actuator inputs. Continuous 3D shape reconstruction is achieved by fitting Bézier curves to the predicted point clouds. Experimental validation demonstrates that our approach achieves high precision, with mean shape estimation errors of 0.08 mm (unloaded) and 0.22 mm (loaded), outperforming state-of-the-art methods in shape sensing for TDCRs. The results validate the efficacy of deep learning-based spatiotemporal data fusion for precise shape estimation under loading conditions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。