动态追踪医学影像数据集及其研究发现,提升算法可靠性。
In the Picture: Medical Imaging Datasets, Artifacts, and their Living Review
- 构建可持续更新的活体文献综述,追踪数据集与新发现
- 发现数据集存在标注偏差、模型捷径等关键问题
- 适合关注数据质量与临床应用可靠性的研究人员
数据集在医学影像研究中至关重要,但标签质量、模型捷径和元数据等问题常被忽视,可能影响算法泛化性并损害患者结果。现有综述多聚焦机器学习方法,且仅覆盖特定应用,且为静态发布,无法反映数据集发布后的新增发现(如偏见、新标注等),我们称这些为研究文物。为此,提出一种持续追踪公共数据集及其研究文物的活体综述,包含监控数据文档文物的框架和可视化引用关系的SQL数据库。还讨论了数据集创建的关键考量、标注最佳实践、模型捷径与人口多样性意义,强调全生命周期管理的重要性。演示系统公开于 http://inthepicture.itu.dk/。
原文摘要 · Abstract (English)
Datasets play a critical role in medical imaging research, yet issues such as label quality, shortcuts, and metadata are often overlooked. This lack of attention may harm the generalizability of algorithms and, consequently, negatively impact patient outcomes. While existing medical imaging literature reviews mostly focus on machine learning (ML) methods, with only a few focusing on datasets for specific applications, these reviews remain static -- they are published once and not updated thereafter. This fails to account for emerging evidence, such as biases, shortcuts, and additional annotations that other researchers may contribute after the dataset is published. We refer to these newly discovered findings of datasets as research artifacts. To address this gap, we propose a living review that continuously tracks public datasets and their associated research artifacts across multiple medical imaging applications. Our approach includes a framework for the living review to monitor data documentation artifacts, and an SQL database to visualize the citation relationships between research artifact and dataset. Lastly, we discuss key considerations for creating medical imaging datasets, review best practices for data annotation, discuss the significance of shortcuts and demographic diversity, and emphasize the importance of managing datasets throughout their entire lifecycle. Our demo is publicly available at http://inthepicture.itu.dk/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。