arXiv:2412.18591cs.CV2024-12

用视觉分割与注意力机制,自动识别胶囊内镜中的出血区域。

ClassifyViStA:WCE Classification with Visual understanding through Segmentation and Attention

  • 融合分割与注意力分支,精准定位出血区域。
  • 在10个数据集上准确率达94.2%,优于传统方法。
  • 结果可解释,适合临床医生辅助诊断使用。

胃肠道(GI)出血是一种严重病症,尤其在医疗资源有限的地区面临巨大诊断挑战。无线胶囊内镜(WCE)已成为可视化胃肠道的重要工具,但其视频需经验丰富的胃肠科医生进行耗时的人工分析,易出错且效率低下。为此,我们提出 ClassifyViStA,一种基于人工智能的框架,用于自动检测和分类 WCE 视频中的出血与非出血帧。该模型包含标准分类路径,并引入两个专用分支:隐式注意力分支聚焦于出血区域,分割分支生成精确的分割掩码,用于分类与可解释性。模型基于 ResNet18 与 VGG16 的集成架构以提升性能。针对出血区域检测,采用 YOLOv8 结合软非极大值抑制(Soft NMS),有效处理重叠边界框,实现更精准、细致的检测。通过分割掩码解释分类结果,使决策过程类似胃肠科医生识别出血区域。本方法不仅实现了胃肠道出血的自动化检测,还提供了可解释性解决方案,减轻医护人员负担,提升诊断效率。代码已公开。

原文摘要 · Abstract (English)

Gastrointestinal (GI) bleeding is a serious medical condition that presents significant diagnostic challenges, particularly in settings with limited access to healthcare resources. Wireless Capsule Endoscopy (WCE) has emerged as a powerful diagnostic tool for visualizing the GI tract, but it requires time-consuming manual analysis by experienced gastroenterologists, which is prone to human error and inefficient given the increasing number of patients.To address this challenge, we propose ClassifyViStA, an AI-based framework designed for the automated detection and classification of bleeding and non-bleeding frames from WCE videos. The model consists of a standard classification path, augmented by two specialized branches: an implicit attention branch and a segmentation branch.The attention branch focuses on the bleeding regions, while the segmentation branch generates accurate segmentation masks, which are used for classification and interpretability. The model is built upon an ensemble of ResNet18 and VGG16 architectures to enhance classification performance. For the bleeding region detection, we implement a Soft Non-Maximum Suppression (Soft NMS) approach with YOLOv8, which improves the handling of overlapping bounding boxes, resulting in more accurate and nuanced detections.The system's interpretability is enhanced by using the segmentation masks to explain the classification results, offering insights into the decision-making process similar to the way a gastroenterologist identifies bleeding regions. Our approach not only automates the detection of GI bleeding but also provides an interpretable solution that can ease the burden on healthcare professionals and improve diagnostic efficiency. Our code is available at ClassifyViStA.

医学影像图像分割可解释AI胶囊内镜

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。