轻量级医学摘要分类用DistilBERT+交叉熵即可，调参后性能更优。
Lightweight Baselines for Medical Abstract Classification: DistilBERT with Cross-Entropy as a Strong Default
- 用DistilBERT加交叉熵损失进行微调，效率高且效果强。
- 通过验证集校准阈值，宏平均F1提升显著，达到最佳部署性能。
- 适合资源受限场景，推荐作为医疗文本分类的默认基线方法。
本研究评估了在预算有限条件下，轻量级医学摘要分类方法的最大性能潜力。在公开的医学摘要数据集上，对BERT base和DistilBERT分别使用交叉熵（CE）、类别加权交叉熵和焦点损失三种目标函数，在相同的分词、序列长度、优化器和学习率调度下进行微调。结果显示，使用原始交叉熵的DistilBERT在原始预测准确率与argmax权衡中表现最强；而经过后处理的校准操作（基于验证集的类间阈值调整）可显著提升实际部署性能，此时焦点损失收益最大。报告了准确率、宏平均F1和加权平均F1指标，发布评估工具，并通过混淆矩阵分析错误结构。实践建议是：优先采用紧凑编码器搭配交叉熵，部署时再引入轻量级校准或阈值调整以提升宏平均平衡性。
原文摘要 · Abstract (English)
The research evaluates lightweight medical abstract classification methods to establish their maximum performance capabilities under financial budget restrictions. On the public medical abstracts corpus, we finetune BERT base and Distil BERT with three objectives cross entropy (CE), class weighted CE, and focal loss under identical tokenization, sequence length, optimizer, and schedule. DistilBERT with plain CE gives the strongest raw argmax trade off, while a post hoc operating point selection (validation calibrated, classwise thresholds) sub stantially improves deployed performance; under this tuned regime, focal benefits most. We report Accuracy, Macro F1, and WeightedF1, release evaluation artifacts, and include confusion analyses to clarify error structure. The practical takeaway is to start with a compact encoder and CE, then add lightweight calibration or thresholding when deployment requires higher macro balance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。