用深度学习统一分析肠镜和病理切片，提升结直肠病诊断效率。
A Multi-Modal Deep Learning Framework for Colorectal Pathology Diagnosis: Integrating Histological and Colonoscopy Data in a Pilot Study
- 构建统一网络，同时处理病理图像与肠镜视频帧。
- 在PathMNIST和HyperKvasir数据集上实现可解释的分类结果。
- 适合临床辅助诊断系统研发者参考。
结直肠疾病（包括炎症和肿瘤）需快速准确诊断以有效治疗。传统流程依赖独立的组织学图像与肠镜视频分析，准备繁琐且易引入偏差。本小规模研究提出一种统一深度学习框架，采用ResNet-50卷积神经网络，同步分类病理切片与肠镜视频帧。使用PathMNIST数据集的静态组织学图像与HyperKvasir数据集的下消化道肠镜视频，结合类别平衡学习、鲁棒增强与校准方法，确保结果准确。该框架实现了可解释、可复现的多模态诊断流程，有助于提升结直肠疾病的检测效率与一致性。
原文摘要 · Abstract (English)
Colorectal diseases, including inflammatory conditions and neoplasms, require quick, accurate care to be effectively treated. Traditional diagnostic pipelines require extensive preparation and rely on separate, individual evaluations on histological images and colonoscopy footage, introducing possible variability and inefficiencies. This pilot study proposes a unified deep learning network that uses convolutional neural networks (CN N s) to classify both histopathological slides and colonoscopy video frames in one pipeline. The pipeline integrates class-balancing learning, robust augmentation, and calibration methods to ensure accurate results. Static colon histology images were taken from the PathMNIST dataset, and the lower gastrointestinal (colonoscopy) videos were drawn from the HyperKvasir dataset. The CNN architecture used was ResNet-50. This study demonstrates an interpretable and reproducible diagnostic pipeline that unifies multiple diagnostic modalities to advance and ease the detection of colorectal diseases.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。