首个支持3D交互式分割的开源工具,可精准分割多种医学影像。
nnInteractive: Redefining 3D Promptable Segmentation
- 基于2D交互生成完整3D分割,支持点、涂画、框和新型套索提示
- 在120+种3D数据集上训练,准确率显著超越现有方法
- 集成至Napari、MITK等主流医学图像平台,临床可用性强
精准高效的3D分割对临床与科研至关重要。尽管SAM等基础模型革新了2D交互分割,但其2D设计与领域偏移限制使其不适用于3D医学图像。现有方法或缺乏体素感知、或交互能力受限、或仅支持少量结构与模态。当前工具也常未集成到主流影像平台,依赖功能受限的Web界面。我们提出nnInteractive,首个全面的3D交互式开集分割方法,支持点、涂画、框及新型套索提示,并通过直观2D交互生成完整3D分割。在120+种多样体数据集(CT、MRI、PET、3D显微镜等)上训练,达到新最优性能。关键突破在于首次集成至Napari、MITK等广泛使用的图像查看器,确保真实场景中临床与科研应用的广泛可及性。大量基准测试表明,nnInteractive远超现有方法,树立了AI驱动交互式3D分割新标准。代码已公开:https://github.com/MIC-DKFZ/napari-nninteractive(Napari插件),https://www.mitk.org/MITK-nnInteractive(MITK集成),https://github.com/MIC-DKFZ/nnInteractive(Python后端)。
原文摘要 · Abstract (English)
Accurate and efficient 3D segmentation is essential for both clinical and research applications. While foundation models like SAM have revolutionized interactive segmentation, their 2D design and domain shift limitations make them ill-suited for 3D medical images. Current adaptations address some of these challenges but remain limited, either lacking volumetric awareness, offering restricted interactivity, or supporting only a small set of structures and modalities. Usability also remains a challenge, as current tools are rarely integrated into established imaging platforms and often rely on cumbersome web-based interfaces with restricted functionality. We introduce nnInteractive, the first comprehensive 3D interactive open-set segmentation method. It supports diverse prompts-including points, scribbles, boxes, and a novel lasso prompt-while leveraging intuitive 2D interactions to generate full 3D segmentations. Trained on 120+ diverse volumetric 3D datasets (CT, MRI, PET, 3D Microscopy, etc.), nnInteractive sets a new state-of-the-art in accuracy, adaptability, and usability. Crucially, it is the first method integrated into widely used image viewers (e.g., Napari, MITK), ensuring broad accessibility for real-world clinical and research applications. Extensive benchmarking demonstrates that nnInteractive far surpasses existing methods, setting a new standard for AI-driven interactive 3D segmentation. nnInteractive is publicly available: https://github.com/MIC-DKFZ/napari-nninteractive (Napari plugin), https://www.mitk.org/MITK-nnInteractive (MITK integration), https://github.com/MIC-DKFZ/nnInteractive (Python backend).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。