用大模型提升科研数据元素与标准映射的准确率。
CDEMapper: Enhancing NIH Common Data Element Normalization using Large Language Models
- 结合检索与大模型生成，自动推荐匹配的CDE标准
- 在四个数据集上，融合GPT的推荐准确率显著提升
- 适合临床研究者快速对齐本地数据与NIH标准
通用数据元素（CDEs）可标准化研究中的数据采集与共享，提升数据互操作性与研究可复现性。然而，由于数据元素范围广、形式多样，实施CDE存在挑战。本文提出CDEMapper，一个基于大语言模型（LLM）的映射工具，用于将本地数据元素映射至美国国立卫生研究院（NIH）CDE标准。该工具包含三个核心模块：（1）对NIH CDE进行索引与嵌入，支持语义搜索；（2）结合Elasticsearch（BM25相似度方法）与先进GPT服务，推荐候选CDE及其允许值；（3）人工审核，用户选择最匹配的CDE与值。评估结果显示，在四个不同数据集上的三种映射场景中，使用GPT嵌入与排序器增强的BM25方法能持续提升映射准确性。CDEMapper提供公开可用、直观易用的界面，集成基础与高级映射功能，采用以用户为中心的设计，形成步骤清晰、质量可控的映射流程。本工作展示了利用大模型辅助CDE推荐与人工校验的潜力，提升了临床研究数据的互操作性，并帮助研究者更清晰识别本地数据与国家标准之间的差距。
原文摘要 · Abstract (English)
Common Data Elements (CDEs) standardize data collection and sharing across studies, enhancing data interoperability and improving research reproducibility. However, implementing CDEs presents challenges due to the broad range and variety of data elements. This study aims to develop an effective and efficient mapping tool to bridge the gap between local data elements and National Institutes of Health (NIH) CDEs. We propose CDEMapper, a large language model (LLM) powered mapping tool designed to assist in mapping local data elements to NIH CDEs. CDEMapper has three core modules: (1) CDE indexing and embeddings. NIH CDEs were indexed and embedded to support semantic search; (2) CDE recommendations. The tool combines Elasticsearch (BM25 similarity methods) with state of the art GPT services to recommend candidate CDEs and their permissible values; and (3) Human review. Users review and select the NIH CDEs and values that best match their data elements and value sets. We evaluate the tool recommendation accuracy against manually annotated mapping results. CDEMapper offers a publicly available, LLM-powered, and intuitive user interface that consolidates essential and advanced mapping services into a streamlined pipeline. It provides a step by step, quality assured mapping workflow designed with a user-centered approach. The evaluation results demonstrated that augmenting BM25 with GPT embeddings and a ranker consistently enhances CDEMapper mapping accuracy in three different mapping settings across four evaluation datasets. This work opens up the potential of using LLMs to assist with CDE recommendation and human curation when aligning local data elements with NIH CDEs. Additionally, this effort enhances clinical research data interoperability and helps researchers better understand the gaps between local data elements and NIH CDEs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。