融合CNN与Transformer，精准识别胶囊内镜中的多种异常
CapsoNet: A CNN-Transformer Ensemble for Multi-Class Abnormality Detection in Video Capsule Endoscopy
- 采用CNN与Transformer联合建模，捕捉局部与全局视觉特征
- 在超5万帧数据上实现86.34%平衡准确率，平均AUC达0.9908
- 适合医学影像分析、智能诊断系统研发人员参考
我们提出CapsoNet，一种为胶囊内镜视觉挑战赛（Capsule Vision 2024）设计的深度学习框架，用于视频胶囊内镜（VCE）图像中多类异常的分类。该模型融合卷积神经网络（CNN）与基于Transformer的架构，以同时捕获局部和全局视觉特征。训练与评估基于包含超过50,000张标注图像的数据集，涵盖十类异常，来自三个公开数据集和一个私有数据集。为应对类别不平衡问题，采用了焦点损失（focal loss）、加权随机采样及大规模数据增强策略。所有模型均经过完整微调以优化集成性能。在官方验证集上，CapsoNet达到86.34%的平衡准确率和0.9908的平均AUC-ROC，使团队Seq2Cure在比赛中获得第5名。代码已开源：http://github.com/arnavs04/capsule-vision-2024。
原文摘要 · Abstract (English)
We present CapsoNet, a deep learning framework developed for the Capsule Vision 2024 Challenge, designed to perform multi-class abnormality classification in video capsule endoscopy (VCE) frames. CapsoNet leverages an ensemble of convolutional neural networks (CNNs) and transformer-based architectures to capture both local and global visual features. The model was trained and evaluated on a dataset of over 50,000 annotated frames spanning ten abnormality classes, sourced from three public and one private dataset. To address the challenge of class imbalance, we employed focal loss, weighted random sampling, and extensive data augmentation strategies. All models were fully fine-tuned to maximize performance within the ensemble. CapsoNet achieved a balanced accuracy of 86.34 percent and a mean AUC-ROC of 0.9908 on the official validation set, securing Team Seq2Cure 5th place in the competition. Our implementation is available at http://github.com/arnavs04/capsule-vision-2024
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。