用普通摄像头还原手术室中每个人的视角,无需额外设备。
Egosurg: Arbitrary view synthesis for egocentric replay of operating room workflows from ambient cameras
- 用稀疏墙挂双目视频重建动态3D场景,结合扩散模型修正画面缺陷。
- 还原视角的清晰度达到PSNR 17.8dB,与真实手持视角接近。
- 适合用于手术复盘、培训和流程分析,支持多角色视角回放。
传统手术观察依赖固定视角或回忆,无法记录影响临床决策的主观视角。环境固定摄像头虽能覆盖整个手术室,但无法还原各成员的实际视野。我们提出EgoSurg框架,仅通过少量墙挂双目视频,即可重建动态手术室场景,并渲染任意角色的主观视角,无需人员装备或干扰手术流程。EgoSurg基于尺度感知的立体深度初始化每帧3D高斯点云表示,并通过图像条件扩散模型优化辅助渲染视图,缓解因摄像机覆盖不足、拥挤和遮挡导致的伪影。我们在四个真实机器人肺科手术及两个模拟全流程会话上评估该框架,覆盖两家医院。近场重建保真度稳定(PSNR 26.8~dB,SSIM .895),合成主观视角与手持式第一视角记录相比达到PSNR 17.8dB、SSIM .766。进一步展示三个应用场景:判定模拟无菌区违规、从角色视角重播手术过程,以及测试人员位置变动的反事实影响。结果表明,现有环境摄像头可转化为可导航的3D手术记录,支持安全事件回溯、培训与工作流分析。
原文摘要 · Abstract (English)
Observing surgical practice has historically relied on fixed vantage points or recollections, leaving the egocentric perspectives that shape clinical decisions undocumented. Ambient fixed cameras capture the operating room (OR) at room scale but cannot recover what any individual team member actually saw. We present EgoSurg, a framework that reconstructs dynamic OR scenes from sparse wall-mounted stereo video and renders arbitrary, role-specific egocentric views without instrumenting personnel or interfering with clinical workflow. EgoSurg initializes a per-timestamp 3D Gaussian Splatting representation from scale-aware stereo depth and refines it with an image-conditioned diffusion model that corrects auxiliary rendered views, mitigating artifacts caused by limited camera coverage, crowding, and occlusion. We evaluated the framework on four real robotic pulmonology procedures and two simulated full-workflow sessions across two hospital sites. Near-field reconstruction fidelity was consistent (PSNR 26.8~dB, SSIM .895) across five workflow phases and both sites, and synthesized egocentric views reached a PSNR of 17.8~dB and an SSIM of .766 against paired hand-held point-of-view recordings. We further demonstrate three case studies for intended use: adjudicating a simulated sterile field violation, replaying a procedure from role-specific viewpoints, and testing a counterfactual change in personnel position. These results indicate that existing ambient camera infrastructure can be turned into a navigable 3D record of surgical work, supporting retrospective review of safety events, training, and workflow analysis from every angle.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。