用轻量微调让大模型精准识别金融文本实体，准确率超90%
Instruction Finetuning DeepSeek-R1-8B Model Using LoRA and NEFTune
- 通过LoRA和NEFTune对DeepSeek-R1-8B进行指令微调
- 在7类金融实体上达到0.912的微平均F1分数
- 适合需要高精度金融信息抽取的研究与应用
金融命名实体识别（NER）对于将非结构化金融报告和新闻转化为结构化知识图谱至关重要。然而，通用大语言模型常出现金融实体误判或忽略领域特定模式的问题。本文研究了基于最新开源模型DeepSeek-R1-8B，结合低秩适应（LoRA）与噪声嵌入微调（NEFTune）的方法用于金融NER。我们构建了包含1693个标注样本的语料库，将每条句子转换为指令-输入-输出三元组。在Transformer层中插入轻量级LoRA矩阵，并在训练时对嵌入向量添加均匀噪声以提升泛化能力。实验表明，采用LoRA的DeepSeek-R1-8B在七类实体（公司、日期、地点、金额、人物、产品、数量）上取得0.901的微平均F1分数，加入NEFTune后进一步提升至0.912，优于Llama3-8B、Qwen3-8B、Baichuan2-7B、T5和BERT-Base等基线模型。
原文摘要 · Abstract (English)
Financial named-entity recognition (NER) is essential for translating unstructured financial reports and news into structured knowledge graphs. However, general-purpose large language models (LLMs) often misclassify financial entities or ignore domain-specific patterns. This paper investigates the use of DeepSeek-R1-8B, a recent open-source large language model, combined with Low-Rank Adaptation (LoRA) and Noisy Embedding Fine-Tuning (NEFTune) for financial NER. Each annotated sentence in our corpus of 1693 samples is converted into an instruction-input-output triple. We insert lightweight LoRA matrices into the Transformer layers and apply NEFTune to improve generalisation by adding uniform noise to embedding vectors during training. Experiments show that the LoRA-adapted DeepSeek-R1-8B achieves a micro-F1 of 0.901 on seven entity types (Company, Date, Location, Money, Person, Product and Quantity), and adding NEFTune further boosts the micro-F1 to 0.912, outperforming Llama3-8B, Qwen3-8B, Baichuan2-7B, T5 and BERT-Base baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。