用结构光+神经网络，从视觉重建推算软组织受力，提升手术机器人触觉反馈精度。
Image-to-Force Estimation for Soft Tissue Interaction in Robotic-Assisted Surgery Using Structured Light
- 通过结构光投影和立体相机捕捉变形，重建软组织高精度三维点云。
- 基于改进PointNet的神经网络，对三种不同硬度硅胶材料实现精准受力估计。
- 无需硬件传感器，适合空间受限的微创手术机器人系统使用。
对于微创手术机器人而言，精确的触觉力反馈对确保与软组织交互的安全性至关重要。然而，由于空间限制，大多数现有系统无法通过硬件传感器直接测量交互力。本文提出一种基于视觉的有效方案：利用一次性结构光投影在软组织上生成特定图案，并结合经训练的图像到力神经网络进行触觉信息处理。从内窥镜立体相机获取的图像被分析以重建软组织变形的高分辨率三维点云。基于此，提出一种改进的基于PointNet的力估计方法，能有效表征软组织复杂的力学特性。在三种不同刚度的硅胶材料上进行了数值力交互实验，结果验证了该方案的有效性。
原文摘要 · Abstract (English)
For Minimally Invasive Surgical (MIS) robots, accurate haptic interaction force feedback is essential for ensuring the safety of interacting with soft tissue. However, most existing MIS robotic systems cannot facilitate direct measurement of the interaction force with hardware sensors due to space limitations. This letter introduces an effective vision-based scheme that utilizes a One-Shot structured light projection with a designed pattern on soft tissue coupled with haptic information processing through a trained image-to-force neural network. The images captured from the endoscopic stereo camera are analyzed to reconstruct high-resolution 3D point clouds for soft tissue deformation. Based on this, a modified PointNet-based force estimation method is proposed, which excels in representing the complex mechanical properties of soft tissue. Numerical force interaction experiments are conducted on three silicon materials with different stiffness. The results validate the effectiveness of the proposed scheme.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。