轻量级模型提升孟加拉语医学实体识别效率与精度
A Lightweight Hybrid Transformer-CRF Architecture for Multi-Type Bangla Medical Entity Recognition
- 用知识蒸馏将12层模型压缩为4层，保留关键特征
- 量化后推理速度提升8.6倍，存储减少近48%
- 针对资源受限场景优化，适合移动医疗应用
医学实体识别（MedER）对从非结构化临床文本中提取结构化信息至关重要。现有基于Transformer的系统计算开销大，难以在资源受限环境中部署。早期研究常使用宽松评估指标，通过高估“外部”（O）类预测来人为提升性能。本文提出一种面向孟加拉语的轻量级医学实体识别框架。构建12层孟加拉语BERT结合条件随机场（CRF）的严格基线模型，用于精确边界识别。通过知识蒸馏将该教师模型压缩为4层学生网络，学生模型学习教师预CRF软发射概率。最终应用INT8动态量化进一步降低模型尺寸与推理成本。量化后学生模型实现8.6倍CPU加速，存储需求比原教师模型减少约48%。
原文摘要 · Abstract (English)
MedER refers to the identification of medical entities. It is crucial for extracting structured clinical information from unstructured medical text. Many existing systems rely on transformer-based models, which are computationally expensive and difficult to deploy in resource-constrained environments. Furthermore, earlier works often use relaxed evaluation metrics that artificially inflate performance by rewarding correct prediction of dominant "Outside" (O) tokens. In this paper, we propose a lightweight Medical Entity Recognition (MedER) framework for the Bangla language. We establish a rigorous baseline using a 12-layer BanglaBERT model combined with a Conditional Random Field (CRF) layer for exact-boundary entity detection. To address deployment constraints, we compress this teacher model into a 4-layer student network through Knowledge Distillation (KD), where the student learns from the teacher's pre-CRF soft emission logits. Finally, we apply INT8 dynamic quantization to further reduce model size and inference cost. Our final quantized student achieves an 8.6x CPU speedup while requiring nearly 48 percent less storage than the CRF teacher model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。