Vision Transformer在耳镜诊断中表现优异，但数据泄漏问题严重影响结果可信度。
Validating Vision Transformers for Otoscopy: Performance and Data-Leakage Effects
- 用Swin Transformer处理耳镜视频帧，基于拉普拉斯和香农熵筛选有效图像
- 原始数据下模型准确率达100%，但修复数据泄漏后降至83%左右
- 提醒医学AI研究必须严格审查数据预处理，避免误判
本研究评估了视觉变压器模型（特别是Swin Transformer）在提升耳部疾病诊断准确性方面的能力，相较于传统卷积神经网络。由于专科耳鼻喉科医生存在27%的误诊率，提升诊断精度至关重要。研究使用智利大学临床医院耳鼻喉科的真实世界数据集，包含多种中耳和外耳病变的耳镜视频。通过拉普拉斯和香农熵阈值选取有效帧，并剔除空白帧。初始实验中，Swin v1与Swin v2分别达到100%和99.1%的准确率，略优于ResNet的99.5%。这些结果优于已有相关研究。然而，评估发现预处理阶段存在严重数据泄漏问题，影响本研究及使用相同原始数据集的其他研究。修复该问题后，模型性能显著下降：修正后的准确率为83%（Swin v1与v2），ResNet为82%。该发现凸显了在医疗机器学习研究中严谨数据处理的重要性。研究表明，尽管视觉变压器具有潜力，但仍需在先进模型架构与有效数据预处理之间取得平衡，才能构建可靠的耳病诊断模型。
原文摘要 · Abstract (English)
This study evaluates the efficacy of vision transformer models, specifically Swin transformers, in enhancing the diagnostic accuracy of ear diseases compared to traditional convolutional neural networks. With a reported 27% misdiagnosis rate among specialist otolaryngologists, improving diagnostic accuracy is crucial. The research utilised a real-world dataset from the Department of Otolaryngology at the Clinical Hospital of the Universidad de Chile, comprising otoscopic videos of ear examinations depicting various middle and external ear conditions. Frames were selected based on the Laplacian and Shannon entropy thresholds, with blank frames removed. Initially, Swin v1 and Swin v2 transformer models achieved accuracies of 100% and 99.1%, respectively, marginally outperforming the ResNet model (99.5%). These results surpassed metrics reported in related studies. However, the evaluation uncovered a critical data leakage issue in the preprocessing step, affecting both this study and related research using the same raw dataset. After mitigating the data leakage, model performance decreased significantly. Corrected accuracies were 83% for both Swin v1 and Swin v2, and 82% for the ResNet model. This finding highlights the importance of rigorous data handling in machine learning studies, especially in medical applications. The findings indicate that while vision transformers show promise, it is essential to find an optimal balance between the benefits of advanced model architectures and those derived from effective data preprocessing. This balance is key to developing a reliable machine learning model for diagnosing ear diseases.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。