提升临床实体识别的精准度,减少误报。
Noise reduction in BERT NER models for clinical entity extraction
- 用概率密度图分析模型置信度,区分强弱预测。
- 在多个临床NER模型上使误报率降低50%至90%。
- 适合对精度要求高的医疗文本分析场景。
临床病历中的实体抽取对精确度要求极高。基于微调的BERT NER模型虽不产生幻觉,但召回表现良好而精确度不足。为此,我们提出一种噪声去除(NR)模型,用于优化NER输出。该模型基于每个标记的概率分数序列,判断其为强或弱预测。传统阈值过滤因SoftMax函数导致高置信度误判而不可靠。因此,我们采用监督建模策略,引入概率密度图(PDM),捕捉Transformer嵌入中语义拉力效应——即在词元序列中各类别预测概率分布的非均匀性。该方法显著提升了分类准确率。实验表明,使用该NR模型后,多个临床NER模型的假阳性率降低了50%至90%。
原文摘要 · Abstract (English)
Precision is of utmost importance in the realm of clinical entity extraction from clinical notes and reports. Encoder Models fine-tuned for Named Entity Recognition (NER) are an efficient choice for this purpose, as they don't hallucinate. We pre-trained an in-house BERT over clinical data and then fine-tuned it for NER. These models performed well on recall but could not close upon the high precision range, needed for clinical models. To address this challenge, we developed a Noise Removal model that refines the output of NER. The NER model assigns token-level entity tags along with probability scores for each token. Our Noise Removal (NR) model then analyzes these probability sequences and classifies predictions as either weak or strong. A naïve approach might involve filtering predictions based on low probability values; however, this method is unreliable. Owing to the characteristics of the SoftMax function, Transformer based architectures often assign disproportionately high confidence scores even to uncertain or weak predictions, making simple thresholding ineffective. To address this issue, we adopted a supervised modeling strategy in which the NR model leverages advanced features such as the Probability Density Map (PDM). The PDM captures the Semantic-Pull effect observed within Transformer embeddings, an effect that manifests in the probability distributions of NER class predictions across token sequences. This approach enables the model to classify predictions as weak or strong with significantly improved accuracy. With these NR models we were able to reduce False Positives across various clinical NER models by 50\% to 90\%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。