金融专用大模型Baichuan4-Finance在专业任务上表现卓越,兼顾通用能力。
Baichuan4-Finance Technical Report
- 基于Baichuan4-Turbo构建金融领域基础模型,采用自约束训练提升数据质量。
- 在金融基准测试中显著优于主流模型,通用能力无明显下降。
- 适合金融场景应用研发,助力行业智能创新。
大型语言模型在语言理解与生成方面表现强劲,但其在金融领域的潜力仍待挖掘。本文报告了Baichuan4-Finance系列模型的开发,包括基于Baichuan4-Turbo基础模型构建的金融领域基础模型Baichuan4-Finance-Base,以及经过监督微调和人类与人工智能反馈强化学习对齐的对话模型Baichuan4-Finance。为提升数据质量,我们设计了完整的数据处理流程,并在持续预训练阶段提出一种新型领域自约束训练策略,使模型在获取金融知识的同时保持通用能力。评估结果显示,Baichuan4-Finance-Base在多数金融任务上显著超越同类基线模型,且在通用基准测试上未出现性能下降。而Baichuan4-Finance在金融认证题与真实场景应用中表现更佳,展现出推动金融大模型领域创新的巨大潜力。
原文摘要 · Abstract (English)
Large language models (LLMs) have demonstrated strong capabilities in language understanding, generation, and reasoning, yet their potential in finance remains underexplored due to the complexity and specialization of financial knowledge. In this work, we report the development of the Baichuan4-Finance series, including a comprehensive suite of foundational Baichuan4-Finance-Base and an aligned language model Baichuan4-Finance, which are built upon Baichuan4-Turbo base model and tailored for finance domain. Firstly, we have dedicated significant effort to building a detailed pipeline for improving data quality. Moreover, in the continual pre-training phase, we propose a novel domain self-constraint training strategy, which enables Baichuan4-Finance-Base to acquire financial knowledge without losing general capabilities. After Supervised Fine-tuning and Reinforcement Learning from Human Feedback and AI Feedback, the chat model Baichuan4-Finance is able to tackle various financial certification questions and real-world scenario applications. We evaluate Baichuan4-Finance on many widely used general datasets and two holistic financial benchmarks. The evaluation results show that Baichuan4-Finance-Base surpasses almost all competitive baselines on financial tasks by significant margins without sacrificing performance on general LLM benchmarks. At the same time, Baichuan4-Finance demonstrates even more impressive performance on financial application scenarios, showcasing its potential to foster community innovation in the financial LLM field.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。