用视觉语言动作框架捕捉外科医生操作指纹,量化个性化带来的隐私风险。
Agentic Surgical AI: Surgeon Style Fingerprinting and Privacy Risk Quantification via Discrete Diffusion in a Vision-Language-Action Framework
- 将手势预测建模为多模态条件下的离散去噪序列任务。
- 在JIGSAWS数据集上实现高精度手势重建与独特行为指纹学习。
- 发现个性特征越强,模型性能越好,但越易泄露身份,需权衡隐私风险。
外科医生的操作风格受训练、经验与运动行为影响而各具特色,但多数手术AI系统忽视这一个性化信号。本文提出一种新型代理式建模方法,用于机器人手术中的外科医生特异性行为预测,结合离散扩散框架与视觉-语言-动作(VLA)流水线。手势预测被建模为结构化序列去噪任务,以手术视频、意图语言及外科医生身份与技能的个性化嵌入作为多模态输入。这些嵌入通过第三方语言模型生成自然语言提示编码,实现个体行为风格保留而不暴露真实身份。在JIGSAWS数据集上的评估表明,该方法能准确重构手势序列,并学习到每个外科医生独特的运动指纹。为量化个性化带来的隐私风险,我们进行成员推断攻击,发现更丰富的嵌入虽提升任务表现,但也显著增加身份泄露风险。结果揭示:个性化虽提升性能,却加剧隐私威胁,强调在手术建模中需平衡个性化与隐私保护。代码已公开于:https://github.com/huixin-zhan-ai/Surgeon_style_fingerprinting。
原文摘要 · Abstract (English)
Surgeons exhibit distinct operating styles shaped by training, experience, and motor behavior-yet most surgical AI systems overlook this personalization signal. We propose a novel agentic modeling approach for surgeon-specific behavior prediction in robotic surgery, combining a discrete diffusion framework with a vision-language-action (VLA) pipeline. Gesture prediction is framed as a structured sequence denoising task, conditioned on multimodal inputs including surgical video, intent language, and personalized embeddings of surgeon identity and skill. These embeddings are encoded through natural language prompts using third-party language models, allowing the model to retain individual behavioral style without exposing explicit identity. We evaluate our method on the JIGSAWS dataset and demonstrate that it accurately reconstructs gesture sequences while learning meaningful motion fingerprints unique to each surgeon. To quantify the privacy implications of personalization, we perform membership inference attacks and find that more expressive embeddings improve task performance but simultaneously increase susceptibility to identity leakage. These findings demonstrate that while personalized embeddings improve performance, they also increase vulnerability to identity leakage, revealing the importance of balancing personalization with privacy risk in surgical modeling. Code is available at: https://github.com/huixin-zhan-ai/Surgeon_style_fingerprinting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。