阿拉伯医学文本分类中，双向编码器比因果解码器更优。
GATech at AbjadMed: Bidirectional Encoders vs. Causal Decoders: Insights from 82-Class Arabic Medical Classification
- 用AraBERTv2结合注意力与均值池化，多样本丢弃增强鲁棒性。
- 在82类细粒度分类中，双向编码器准确率显著高于因果解码器。
- 适合阿拉伯语医疗NLP、需精准语义理解的场景使用。
本文针对82个类别阿拉伯医学文本分类任务，提出基于微调AraBERTv2编码器的系统方案，结合注意力与均值混合池化策略及多样本丢弃进行正则化。系统对比了多种多语言和阿拉伯语专用编码器，以及若干大规模因果解码器（包括基于Llama 3.3 70B的零样本重排序和Qwen 3B隐藏状态特征提取）。结果表明，专门的双向编码器在捕捉细粒度医学文本语义边界方面显著优于因果解码器；后者因优化目标为下一个词预测，生成序列偏倚嵌入，难以胜任分类任务。尽管训练数据存在显著类别不平衡与标签噪声，微调编码器仍展现出优越的语义压缩能力。最终测试集性能指标如准确率与宏观F1得分均被报告并分析。
原文摘要 · Abstract (English)
This paper presents system description for Arabic medical text classification across 82 distinct categories. Our primary architecture utilizes a fine-tuned AraBERTv2 encoder enhanced with a hybrid pooling strategies, combining attention and mean representations, and multi-sample dropout for robust regularization. We systematically benchmark this approach against a suite of multilingual and Arabic-specific encoders, as well as several large-scale causal decoders, including zero-shot re-ranking via Llama 3.3 70B and feature extraction from Qwen 3B hidden states. Our findings demonstrate that specialized bidirectional encoders significantly outperform causal decoders in capturing the precise semantic boundaries required for fine-grained medical text classification. We show that causal decoders, optimized for next-token prediction, produce sequence-biased embeddings that are less effective for categorization compared to the global context captured by bidirectional attention. Despite significant class imbalance and label noise identified within the training data, our results highlight the superior semantic compression of fine-tuned encoders for specialized Arabic NLP tasks. Final performance metrics on the test set, including Accuracy and Macro-F1, are reported and discussed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。