用混合现实技术让病理医生沉浸式查看超大病理图，减少操作负担。
Beyond the Monitor: Mixed Reality Visualization and Multimodal AI for Enhanced Digital Pathology Workflow
- 通过眼动、手势和语音实现对超大病理图的自然交互
- 集成图像检索与对话助手，支持实时诊断辅助
- 适合需要高效处理高分辨率病理数据的临床研究者
病理科医生诊断癌症依赖于百万像素级全切片图像（WSIs），但当前数字工作流程分散。这些多尺度数据常超过10万×10万像素，而标准2D显示器限制了视野，迫使频繁拖动和缩放，增加认知负荷并打断诊断节奏。我们提出PathVis，一个基于Apple Vision Pro的混合现实平台，将整个生态系统整合至单一沉浸式环境。PathVis以眼动、自然手势和语音命令取代传统鼠标导航，实现对巨幅数据的具身化探索。系统融合多模态AI代理：基于内容的图像检索引擎可空间化展示相似患者病例，便于预后对比；对话式助手提供实时解读。通过融合沉浸式可视化与集成AI能力，PathVis在简化诊断流程、减少上下文切换负担方面展现潜力。本文介绍系统架构，并通过初步定性评估验证平台可行性。PathVis源码及演示视频已公开：https://github.com/jaiprakash1824/Path_Vis。
原文摘要 · Abstract (English)
Pathologists diagnose cancer using gigapixel whole-slide images (WSIs), but the current digital workflow is fragmented. These multiscale datasets often exceed 100,000 x 100,000 pixels, yet standard 2D monitors restrict the field of view. This disparity forces constant panning and zooming, which increases cognitive load and disrupts diagnostic momentum. We introduce PathVis, a mixed-reality platform for Apple Vision Pro that unifies this ecosystem into a single immersive environment. PathVis replaces indirect mouse navigation with embodied interaction, utilizing eye gaze, natural hand gestures, and voice commands to explore gigapixel data. The system integrates multimodal AI agents to support computer-aided diagnosis: a content-based image retrieval engine spatially displays similar patient cases for side-by-side prognostic comparison, while a conversational assistant provides real-time interpretation. By merging immersive visualization with integrated AI capabilities, PathVis shows promise in streamlining diagnostic workflows and mitigating the burden of context switching. This paper presents the system architecture and a preliminary qualitative evaluation demonstrating the platform's feasibility. The PathVis source code and a demo video are publicly available at: https://github.com/jaiprakash1824/Path_Vis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。