arXiv:2605.04408cs.RO2026-05

统一多模态术中信息,实现腹腔镜机器人的自主稳定控制。

Autonomous Laparoscope Control through Unified Mechanics-Based Representation of Multimodal Intraoperative Information

  • 将位置、力/力矩、图像等多源信号映射为统一力矩表示
  • 支持在保持手术机器人转动中心约束下,自动追踪器械并减小穿刺点受力
  • 适用于需要高稳定性与安全性的智能手术辅助系统

腹腔镜持镜机器人可为外科医生提供稳定的术中视野,减轻人力负担。为维持理想视野,机器人需根据术中多模态信号持续调整镜头姿态。然而,位置、力/力矩和图像等信号在物理意义和量纲上差异显著,难以构建统一表征,也难以生成可直接用于控制的指令。为此,我们提出一种基于统一力学建模的腹腔镜控制方法:首先设计多源信号(位置、力/力矩、图像)的映射策略,将其统一为操作空间中的等效力矩表示;随后采用任务优先级机制,分别将力矩注入任务空间与零空间,并通过任务优先投影合成控制指令,实现多模态信息的统一表征与协同融合。最后,以术中远程中心(RCM)位置、力/力矩传感器读数和腹腔镜图像为例,构建了RCM约束力矩以满足几何约束、减少穿刺点载荷,以及器械跟踪力矩以实现自主视觉追踪。在手术模拟体和活体猪实验中,该方法成功实现了包括柔顺操作和自主器械追踪在内的多任务运行,同时保持了RCM约束并降低了持续性穿刺点受力。

原文摘要 · Abstract (English)

Laparoscope-holding robots can provide surgeons with a stable laparoscopic field of view (FOV) and reduce the burden on human assistants. To maintain an ideal intraoperative FOV, the robot must continuously adjust the laparoscope pose according to intraoperative information. However, intraoperative multimodal signals, such as position, force/torque, and images, differ markedly in physical meaning and units, making it difficult to build a unified representation and to generate control commands that can be used directly for laparoscope control. To address this issue, we propose a laparoscope-holding robot control method based on unified mechanics modeling of multimodal information. First, we design mapping strategies for multiple intraoperative sources, including position, force/torque, and images, and unify them into an equivalent-wrench representation in the operational space. Then, using a task-priority scheme, we inject the wrenches into the task space and the null space, respectively, and synthesize laparoscope control commands via task-priority projection, thereby achieving consistent representation and coordinated fusion of multimodal information within a single framework. Finally, taking the intraoperative remote center of motion (RCM) position, force/torque sensor readings, and laparoscopic images as examples, we construct an RCM-constraint wrench to enforce the RCM geometric constraint and reduce the contact force at the trocar site, a laparoscope-manipulation wrench to enable compliant dragging, and an instrument-tracking wrench to achieve autonomous visual tracking of the instruments. Experiments on a surgical phantom and in vivo porcine trials demonstrate that the proposed method supports multi-task operation, including compliant laparoscope manipulation and autonomous instrument tracking, while maintaining the RCM constraint and reducing sustained trocar-site loading.

手术机器人多模态融合力控自主跟踪

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。