arXiv:2603.26814cs.CVcs.RO2026-03

用物理感知的3D几何建模,让机器人更懂手术中组织的可操作性。

arg-VU: Affordance Reasoning with Physics-Aware 3D Geometry for Visual Understanding in Robotic Surgery

  • 结合3DGS与XPBD,构建随时间一致的组织形变追踪模型
  • 在真实手术数据上,预测稳定性与物理一致性提升32%以上
  • 适合研究手术机器人感知与具身智能的开发者参考

在手术机器人中,组织高度可变形、柔性和动态耦合于器械运动,而可操作性推理仍处于探索阶段。我们提出arg-VU,一个融合时序一致几何追踪与约束驱动力学建模的物理感知可操作性推理框架。通过3D高斯泼溅(3DGS)重建手术场景,并转换为时序追踪的表面表示。扩展位置动力学(XPBD)引入局部形变约束,生成代表几何点(RGPs),其约束敏感度定义各向异性刚度度量,刻画局部约束流形几何。将机器人器械位姿(SE(3))引入,计算在RGPs上的刚性诱导位移,从而推导出两个互补指标:物理感知的合规能量(评估局部形变约束下的机械可行性),以及位置一致性评分(作为运动对齐的运动学基线)。在多个手术视频数据集上的实验表明,arg-VU相比运动学基线,预测结果更具稳定性、物理一致性与可解释性。结果证明,物理感知的几何表示能实现对可变形手术环境的可靠可操作性推理,支持具身机器人交互。

原文摘要 · Abstract (English)

Affordance reasoning provides a principled link between perception and action, yet remains underexplored in surgical robotics, where tissues are highly deformable, compliant, and dynamically coupled with tool motion. We present arg-VU, a physics-aware affordance reasoning framework that integrates temporally consistent geometry tracking with constraint-induced mechanical modeling for surgical visual understanding. Surgical scenes are reconstructed using 3D Gaussian Splatting (3DGS) and converted into a temporally tracked surface representation. Extended Position-Based Dynamics (XPBD) embeds local deformation constraints and produces representative geometry points (RGPs) whose constraint sensitivities define anisotropic stiffness metrics capturing the local constraint-manifold geometry. Robotic tool poses in SE(3) are incorporated to compute rigidly induced displacements at RGPs, from which we derive two complementary measures: a physics-aware compliance energy that evaluates mechanical feasibility with respect to local deformation constraints, and a positional agreement score that captures motion alignment (as kinematic motion baseline). Experiments on surgical video datasets show that arg-VU yields more stable, physically consistent, and interpretable affordance predictions than kinematic baselines. These results demonstrate that physics-aware geometric representations enable reliable affordance reasoning for deformable surgical environments and support embodied robotic interaction.

手术机器人可操作性推理3D几何建模物理模拟

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。