KodeXv0.1在金融问答中超越GPT-4，8B模型性能领先9.24%。
KodeXv0.1: A Family of State-of-the-Art Financial Large Language Models
- 用财报和电话会数据构建金融专用三元组训练集，适配Llama 3.1基座模型。
- 8B版本比同规模指令模型高9.24%，70B版全面超越GPT-4。
- 适合金融分析、投资决策等专业场景，尤其看重模型可靠性的人群。
当前先进大模型难以满足高度专业化领域需求。我们提出KodeXv0.1，一个在金融问答任务上优于GPT-4的大型语言模型家族。基于Llama 3.1 8B与70B基座模型，通过自定义训练流程，利用公开财务文档（如财报、业绩电话会）构建高质量合成数据集，生成包含上下文-问题-答案的三元组。使用该数据集的训练子集，对基座模型进行4bit LoRA指令微调，得到KodeX-8Bv0.1与KodeX-70Bv0.1。在FinanceBench、FinQABench及预留测试集上评估表明：KodeX-8Bv0.1在同参数量级下比前沿指令模型更可靠，最高提升9.24%；甚至可超越GPT-4达7.07%。KodeX-70Bv0.1进一步超越，所有基准测试均优于GPT-4。
原文摘要 · Abstract (English)
Although powerful, current cutting-edge LLMs may not fulfil the needs of highly specialised sectors. We introduce KodeXv0.1, a family of large language models that outclass GPT-4 in financial question answering. We utilise the base variants of Llama 3.1 8B and 70B and adapt them to the financial domain through a custom training regime. To this end, we collect and process a large number of publicly available financial documents such as earnings calls and business reports. These are used to generate a high-quality, synthetic dataset consisting of Context-Question-Answer triplets which closely mirror real-world financial tasks. Using the train split of this dataset, we perform RAG-aware 4bit LoRA instruction tuning runs of Llama 3.1 base variants to produce KodeX-8Bv0.1 and KodeX-70Bv0.1. We then complete extensive model evaluations using FinanceBench, FinQABench and the withheld test split of our dataset. Our results show that KodeX-8Bv0.1 is more reliable in financial contexts than cutting-edge instruct models in the same parameter regime, surpassing them by up to 9.24%. In addition, it is even capable of outperforming state-of-the-art proprietary models such as GPT-4 by up to 7.07%. KodeX-70Bv0.1 represents a further improvement upon this, exceeding GPT-4's performance on every tested benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。