轻量医疗NLP模型助力弱势群体平等获取诊疗支持
A Patient-Doctor-NLP-System to contest inequality for less privileged
- 融合蒸馏与频域调制的紧凑架构，降低计算开销
- 在印地语和视障场景下性能接近主流模型，但资源消耗更低
- 适合农村医疗、低资源语言及无障碍应用
迁移学习推动了大语言模型在主流自然语言处理中的快速发展，但在资源受限的真实医疗场景中训练和部署大型语言模型仍具挑战。本研究针对视障用户及印地语等低资源语言使用者在农村地区就医难的问题，提出PDFTEMRA（高效蒸馏频域变换器集成模型带随机激活）——一种基于紧凑变换器架构的模型，结合模型蒸馏、频域调制、集成学习与随机激活模式，在显著降低计算成本的同时保持语言理解能力。该模型在专为印地语和可访问性设计的医疗问答与咨询数据集上训练与评估，并与主流NLP模型基线进行对比。结果表明，PDFTEMRA在性能上可与先进模型媲美，但计算需求大幅减少，证明其适用于可访问、包容性且资源有限的医疗自然语言处理应用。
原文摘要 · Abstract (English)
Transfer Learning (TL) has accelerated the rapid development and availability of large language models (LLMs) for mainstream natural language processing (NLP) use cases. However, training and deploying such gigantic LLMs in resource-constrained, real-world healthcare situations remains challenging. This study addresses the limited support available to visually impaired users and speakers of low-resource languages such as Hindi who require medical assistance in rural environments. We propose PDFTEMRA (Performant Distilled Frequency Transformer Ensemble Model with Random Activations), a compact transformer-based architecture that integrates model distillation, frequency-domain modulation, ensemble learning, and randomized activation patterns to reduce computational cost while preserving language understanding performance. The model is trained and evaluated on medical question-answering and consultation datasets tailored to Hindi and accessibility scenarios, and its performance is compared against standard NLP state-of-the-art model baselines. Results demonstrate that PDFTEMRA achieves comparable performance with substantially lower computational requirements, indicating its suitability for accessible, inclusive, low-resource medical NLP applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。