研究临床文本分类中训练数据量与词汇特征的关系，发现600样本可接近最优性能。
Determinants of Training Corpus Size for Clinical Text Classification
- 用BERT+随机森林在MIMIC-III上测试不同数据量下的分类表现。
- 600文档即可达到10000文档95%的性能，且学习曲线差异显著。
- 强预测词多、噪声词少的任务收敛更快，每100个噪声词降0.02准确率。
临床文本分类需足够训练数据以达最优性能，通常标注200-500份文档，但此数量缺乏依据且未关联词汇特性。本研究基于公开的MIMIC-III数据集（含出院记录与ICD-9诊断标签），采用预训练BERT嵌入结合随机森林分类器，对10个随机选择的诊断任务，从100到10,000份文档变化训练集规模，通过袋装词模型上的Lasso逻辑回归识别强预测词与噪声词。结果显示，尽管使用相同预处理和算法，10项任务的学习曲线差异显著；所有任务中，600份文档即能达到10,000份文档95%的性能。词汇分析表明，强预测词越多、噪声词越少，学习曲线越陡峭：每增加100个噪声词，准确率下降约0.02；每增加100个强预测词，最大准确率提升约0.04。
原文摘要 · Abstract (English)
Introduction: Clinical text classification using natural language processing (NLP) models requires adequate training data to achieve optimal performance. For that, 200-500 documents are typically annotated. The number is constrained by time and costs and lacks justification of the sample size requirements and their relationship to text vocabulary properties. Methods: Using the publicly available MIMIC-III dataset containing hospital discharge notes with ICD-9 diagnoses as labels, we employed pre-trained BERT embeddings followed by Random Forest classifiers to identify 10 randomly selected diagnoses, varying training corpus sizes from 100 to 10,000 documents, and analyzed vocabulary properties by identifying strong and noisy predictive words through Lasso logistic regression on bag-of-words embeddings. Results: Learning curves varied significantly across the 10 classification tasks despite identical preprocessing and algorithms, with 600 documents sufficient to achieve 95% of the performance attainable with 10,000 documents for all tasks. Vocabulary analysis revealed that more strong predictors and fewer noisy predictors were associated with steeper learning curves, where every 100 additional noisy words decreased accuracy by approximately 0.02 while 100 additional strong predictors increased maximum accuracy by approximately 0.04.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。