arXiv:2512.07552cs.CL2025-12被引 1

AI系统SafeTerm自动匹配药物不良反应术语,准确率超89%。

Performance of the SafeTerm AI-Based MedDRA Query System Against Standardised MedDRA Queries

  • 用向量相似度与聚类算法自动检索医学术语,按相关性排序。
  • 在0.70阈值下,召回率达48%,精确率达45%,表现稳定。
  • 适合药企和监管机构用于自动化安全信号检测,提升效率。

在新药上市前安全性评估中,将相关不良事件术语归入标准医学术语集(SMQs)或开放性医学术语集(OCMQs)对信号检测至关重要。本文评估SafeTerm自动医学查询(AMQ)系统在MedDRA SMQs上的性能。该系统为基于人工智能的定量系统,能理解医学术语并自动检索相关首选术语(PTs),通过多准则统计方法生成0-1范围的相关性评分。系统将查询词与MedDRA PTs嵌入多维向量空间,利用余弦相似度和极值聚类生成排序列表。验证基于110个一级SMQs(v28.1)。在中等相似度阈值下,召回率达94%,显示良好检索敏感性;更高阈值可提升精确度至89%。最优阈值0.70时,整体召回率为48%,精确率为45%。限定窄义术语时,于0.75阈值下性能略优。自动阈值选择法(0.66)更侧重召回(0.58)而非精确度(0.29)。SafeTerm AMQ在SMQs及经清洗的OCMQs上表现相当满意,可作为自动化医学术语生成的补充工具,平衡召回与精确度。建议在查询构建中使用恰当的MedDRA PT术语,并采用自动阈值法以优化召回率。提高相似度得分有助于筛选更精确的窄义术语。

原文摘要 · Abstract (English)

In pre-market drug safety review, grouping related adverse event terms into SMQs or OCMQs is critical for signal detection. We assess the performance of SafeTerm Automated Medical Query (AMQ) on MedDRA SMQs. The AMQ is a novel quantitative artificial intelligence system that understands and processes medical terminology and automatically retrieves relevant MedDRA Preferred Terms (PTs) for a given input query, ranking them by a relevance score (0-1) using multi-criteria statistical methods. The system (SafeTerm) embeds medical query terms and MedDRA PTs in a multidimensional vector space, then applies cosine similarity, and extreme-value clustering to generate a ranked list of PTs. Validation was conducted against tier-1 SMQs (110 queries, v28.1). Precision, recall and F1 were computed at multiple similarity-thresholds, defined either manually or using an automated method. High recall (94%)) is achieved at moderate similarity thresholds, indicative of good retrieval sensitivity. Higher thresholds filter out more terms, resulting in improved precision (up to 89%). The optimal threshold (0.70)) yielded an overall recall of (48%) and precision of (45%) across all 110 queries. Restricting to narrow-term PTs achieved slightly better performance at an increased (+0.05) similarity threshold, confirming increased relatedness of narrow versus broad terms. The automatic threshold (0.66) selection prioritizes recall (0.58) to precision (0.29). SafeTerm AMQ achieves comparable, satisfactory performance on SMQs and sanitized OCMQs. It is therefore a viable supplementary method for automated MedDRA query generation, balancing recall and precision. We recommend using suitable MedDRA PT terminology in query formulation and applying the automated threshold method to optimise recall. Increasing similarity scores allows refined, narrow terms selection.

AI医疗药物安全术语匹配

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。