用分角色专家模型提升金融分析能力,效果优于同规模大模型。
FinTeamExperts: Role Specialized MOEs For Financial Analysis
- 分角色训练三类专家模型:宏观、微观、量化分析师
- 在四组金融数据集上,三项超越同尺寸模型,一项表现最优
- 适合需要多角度金融研判的机构与研究者使用
大型语言模型(如ChatGPT、Phi3和Llama-3)在无需微调的情况下即可泛化知识,推动AI发展。然而其在金融领域的应用仍有限。金融领域复杂,需涵盖宏观、微观经济趋势及量化分析等多维度理解。为此,我们提出FinTeamExperts——一种基于混合专家(MOEs)架构的角色专业化金融分析框架。该框架通过训练三个80亿参数模型分别专精于宏观分析师、微观分析师和量化分析师角色,模拟团队协作。各模型在不同语料库上训练,并在下游任务上进行指令微调以对齐实际金融需求。实验表明,在四个数据集中的三个上,FinTeamExperts超越同尺寸及其他更大模型;在第四项更复杂任务中,亦胜过所有同尺寸模型。验证了角色专业化与持续训练的有效性。
原文摘要 · Abstract (English)
Large Language Models (LLMs), such as ChatGPT, Phi3 and Llama-3, are leading a significant leap in AI, as they can generalize knowledge from their training to new tasks without fine-tuning. However, their application in the financial domain remains relatively limited. The financial field is inherently complex, requiring a deep understanding across various perspectives, from macro, micro economic trend to quantitative analysis. Motivated by this complexity, a mixture of expert LLMs tailored to specific financial domains could offer a more comprehensive understanding for intricate financial tasks. In this paper, we present the FinTeamExperts, a role-specialized LLM framework structured as a Mixture of Experts (MOEs) for financial analysis. The framework simulates a collaborative team setting by training each model to specialize in distinct roles: Macro Analysts, Micro analysts, and Quantitative Analysts. This role-specific specialization enhances the model's ability to integrate their domain-specific expertise. We achieve this by training three 8-billion parameter models on different corpus, each dedicated to excelling in specific finance-related roles. We then instruct-tune FinTeamExperts on downstream tasks to align with practical financial tasks. The experimental results show that FinTeamExperts outperform all models of the same size and larger on three out of four datasets. On the fourth dataset, which presents a more complex task, FinTeamExperts still surpass all models of the same size. This highlights the success of our role-based specialization approach and the continued training approach for FinTeamExperts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。