arXiv:2411.02730cs.CLcs.LG2024-11被引 5

用大模型自动匹配医学数据变量,提升跨研究数据整合效率

A Natural Language Processing Approach to Support Biomedical Data Harmonization: Leveraging Large Language Models

  • 结合大模型与随机森林,通过语义相似度匹配变量
  • 集成方法在30个候选中命中率98%,均值倒数排名0.73
  • 适合需要整合多源医学数据的研究者快速对齐变量

生物医学研究需大规模多样样本以避免偏倚。自动化变量匹配可加速数据整合。现有方法多依赖词汇或本体匹配,进展有限。本文利用欧洲与日本两个GERAS队列研究数据,构建了352个欧盟变量与1322个日本候选变量的匹配数据集(正例为人工标注匹配对,负例为未匹配对)。基于数据字典中的变量标签与定义,开发并评估两类自然语言处理方法:(1)基于大语言模型(LLM)的方法;(2)模糊匹配方法。进一步采用随机森林(RF)集成学习,融合各NLP方法输出,在50次随机4:1训练测试划分中优化超参数并验证性能。对每个欧盟变量,按相似度或概率得分对1322个日本变量排序,使用顶n命中率(HRn)和均值倒数排名(MRR)评估。结果显示,单一方法中E5表现最优,达HR-30=0.90、MRR=0.70;而集成方法在全部指标上显著优于E5(P<0.001),平均达到HR-30=0.98、MRR=0.73,其中大模型特征贡献最大。主要错误原因在于数据字典中变量定义模糊。

原文摘要 · Abstract (English)

Biomedical research requires large, diverse samples to produce unbiased results. Automated methods for matching variables across datasets can accelerate this process. Research in this area has been limited, primarily focusing on lexical matching and ontology based semantic matching. We aimed to develop new methods, leveraging large language models (LLM) and ensemble learning, to automate variable matching. Methods: We utilized data from two GERAS cohort (European and Japan) studies to develop variable matching methods. We first manually created a dataset by matching 352 EU variables with 1322 candidate JP variables, where matched variable pairs were positive and unmatched pairs were negative instances. Using this dataset, we developed and evaluated two types of natural language processing (NLP) methods, which matched variables based on variable labels and definitions from data dictionaries: (1) LLM-based and (2) fuzzy matching. We then developed an ensemble-learning method, using the Random Forest model, to integrate individual NLP methods. RF was trained and evaluated on 50 trials. Each trial had a random split (4:1) of training and test sets, with the model's hyperparameters optimized through cross-validation on the training set. For each EU variable, 1322 candidate JP variables were ranked based on NLP-derived similarity scores or RF's probability scores, denoting their likelihood to match the EU variable. Ranking performance was measured by top-n hit ratio (HRn) and mean reciprocal rank (MRR). Results:E5 performed best among individual methods, achieving 0.90 HR-30 and 0.70 MRR. RF performed better than E5 on all metrics over 50 trials (P less than 0.001) and achieved an average HR 30 of 0.98 and MRR of 0.73. LLM-derived features contributed most to RF's performance. One major cause of errors in automatic variable matching was ambiguous variable definitions within data dictionaries.

大模型数据整合生物医学变量匹配

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。