10亿参数小模型助力日本临床医学文本生成与理解
Technical Report: Small Language Model for Japanese Clinical and Medicine
- 基于高质量日文医疗文本训练，专化分词器提升处理精度
- 在8项任务中6项超越大模型，证明小模型可行
- 适合医疗领域轻量化应用，推动日本临床AI发展
本报告介绍一款面向日文临床与医学领域的10亿参数小型语言模型（NCVC-slm-1），该模型使用高质量日文文本进行训练，并通过专门设计的预处理流程、专用形态分析器和分词器增强临床医学内容覆盖，包括疾病、药物与检查项目。该模型不仅具备文本生成能力，还展现出理解临床医学文本的可行性。在与多个大模型对比中，经微调后的NCVC-slm-1在JMED-LLM数据集的8项任务中取得了6项最高分，验证了小模型在临床医学下游任务中的有效性。研究结果表明，该模型具有在医疗领域实现轻量化部署的潜力，有望推动日本临床医学人工智能的发展。
原文摘要 · Abstract (English)
This report presents a small language model (SLM) for Japanese clinical and medicine, named NCVC-slm-1. This 1B parameters model was trained using Japanese text classified to be of high-quality. Moreover, NCVC-slm-1 was augmented with respect to clinical and medicine content that includes the variety of diseases, drugs, and examinations. Using a carefully designed pre-processing, a specialized morphological analyzer and tokenizer, this small and light-weight model performed not only to generate text but also indicated the feasibility of understanding clinical and medicine text. In comparison to other large language models, a fine-tuning NCVC-slm-1 demonstrated the highest scores on 6 tasks of total 8 on JMED-LLM. According to this result, SLM indicated the feasibility of performing several downstream tasks in the field of clinical and medicine. Hopefully, NCVC-slm-1 will be contributed to develop and accelerate the field of clinical and medicine for a bright future.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。