用视觉查询定位胎儿超声标准切面,提升筛查效率与一致性。
MCAT: Visual Query-Based Localization of Standard Anatomical Clips in Fetal Ultrasound Videos Using Multi-Tier Class-Aware Token Transformer
- 通过多层级感知令牌的视觉查询方法,精准定位视频中的标准解剖切面。
- 在超声数据集上达到10%和13%的mIoU提升,且仅用96%更少的令牌。
- 适合需要快速、准确获取标准切面的产科医生,尤其利于资源匮乏地区。
胎儿超声视频中准确获取标准切面对于胎儿生长评估、异常检测及遵循临床指南至关重要。然而,手动选择标准帧耗时且易受操作者间差异影响。现有方法多依赖图像级方法,仅捕捉标准帧后分类不同解剖结构,忽略了视频采集与解读的动态性。为此,本文提出多层级类感知令牌变换器(MCAT),一种基于视觉查询的视频片段定位(VQ-VCL)方法,使超声医师只需快速扫查,再提供目标解剖结构的视觉查询,即可返回包含标准帧的视频片段,便于全面筛查潜在异常。我们在两个超声视频数据集及基于Ego4D的自然图像VQ-VCL数据集上评估了MCAT,其在超声数据集上分别比最先进方法提升10%和13%的mIoU,Ego4D数据集上提升5.35%,同时仅使用96%更少的令牌。该方法的高效与高精度对公共健康具有重要意义,尤其在低收入和中等收入国家(LMICs),可优化产前护理,简化超声筛查与诊断流程,使超声医师能检查更多患者。
原文摘要 · Abstract (English)
Accurate standard plane acquisition in fetal ultrasound (US) videos is crucial for fetal growth assessment, anomaly detection, and adherence to clinical guidelines. However, manually selecting standard frames is time-consuming and prone to intra- and inter-sonographer variability. Existing methods primarily rely on image-based approaches that capture standard frames and then classify the input frames across different anatomies. This ignores the dynamic nature of video acquisition and its interpretation. To address these challenges, we introduce Multi-Tier Class-Aware Token Transformer (MCAT), a visual query-based video clip localization (VQ-VCL) method, to assist sonographers by enabling them to capture a quick US sweep. By then providing a visual query of the anatomy they wish to analyze, MCAT returns the video clip containing the standard frames for that anatomy, facilitating thorough screening for potential anomalies. We evaluate MCAT on two ultrasound video datasets and a natural image VQ-VCL dataset based on Ego4D. Our model outperforms state-of-the-art methods by 10% and 13% mIoU on the ultrasound datasets and by 5.35% mIoU on the Ego4D dataset, using 96% fewer tokens. MCAT's efficiency and accuracy have significant potential implications for public health, especially in low- and middle-income countries (LMICs), where it may enhance prenatal care by streamlining standard plane acquisition, simplifying US-based screening, diagnosis and allowing sonographers to examine more patients.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。