用混合精度训练加速孟加拉语NLP，省资源不丢性能。
Accelerating Bangla NLP Tasks with Automatic Mixed Precision: Resource-Efficient Training Preserving Model Efficacy
- 采用动态16/32位浮点计算，降低显存占用并提速训练。
- 训练速度提升44.5%，显存减少17.6%，性能损失小于0.3%。
- 适合硬件受限的低资源语言研究者使用。
自然语言处理模型训练需大量计算资源与时间，尤其在孟加拉语这类资源有限的语言上更为严峻。本文探索自动混合精度（AMP）训练方法，在不牺牲模型性能的前提下提升计算效率。通过动态使用16位与32位浮点运算，显著降低GPU内存需求并加快训练速度。我们在四种标准孟加拉语任务（情感分析、命名实体识别、错误分类、问答）上，使用四个基于Transformer的模型（BanglaBERT、BanglishBERT、XLM-R、mBERT）进行评估。结果表明，AMP使训练速度提升44.5%，内存消耗减少17.6%，同时F-1分数保持在全精度基线的99.7%以上。该实证研究证明，AMP可有效降低高算力门槛，助力低资源环境下实现先进NLP能力。
原文摘要 · Abstract (English)
Training models for Natural Language Processing (NLP) requires substantial computational resources and time, posing significant challenges, especially for NLP development in Bangla, where access to high-end hardware is often limited. In this work, we explore automatic mixed precision (AMP) training as a means to improve computational efficiency without sacrificing model performance. By leveraging a dynamic mix of 16-bit and 32-bit floating-point computations, AMP lowers GPU memory requirements and speeds up training without degrading model performance. We evaluate AMP across four standard Bangla NLP tasks, namely sentiment analysis, named entity recognition, error classification, and question answering, using four transformer-based models: BanglaBERT, BanglishBERT, XLM-R, and mBERT. Our results demonstrate that AMP accelerates training by 44.5% and reduces memory consumption by 17.6%, while maintaining F-1 score within 99.7% of the full-precision baselines. This empirical study highlights AMP's potential to democratize access to state-of-the-art NLP capabilities in hardware-constrained settings by lowering computational barriers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。