首个面向金融谣言检测的开源大模型，用指令微调提升识别准确率。
FMDLlama: Financial Misinformation Detection based on Large Language Models
- 基于Llama3.1微调，构建专用金融谣言检测指令数据集
- 在多任务评测中超越开源及OpenAI主流模型表现
- 适合金融风控、舆情监测等需要快速判断信息真伪的场景
社交媒体的兴起使虚假信息传播更加迅速。在金融领域，信息准确性关乎市场稳定，因此金融谣言检测（FMD）成为亟待解决的问题。尽管大语言模型（LLMs）在多个领域表现出色，但现有研究仍依赖传统方法，未充分探索LLMs在FMD中的应用，主要受限于缺乏FMD指令微调数据集与评估基准。本文提出FMDLlama，首个基于Llama3.1指令微调的开源金融谣言检测大模型；构建首个多任务FMD指令数据集（FMDID），支持模型指令学习；并设计综合性评估基准（FMD-B），涵盖分类与解释生成任务，全面测试模型性能。在FMD-B上对比多种LLM，FMDLlama在开放源码模型中领先，甚至优于OpenAI产品。项目已开源：https://github.com/lzw108/FMD。
原文摘要 · Abstract (English)
The emergence of social media has made the spread of misinformation easier. In the financial domain, the accuracy of information is crucial for various aspects of financial market, which has made financial misinformation detection (FMD) an urgent problem that needs to be addressed. Large language models (LLMs) have demonstrated outstanding performance in various fields. However, current studies mostly rely on traditional methods and have not explored the application of LLMs in the field of FMD. The main reason is the lack of FMD instruction tuning datasets and evaluation benchmarks. In this paper, we propose FMDLlama, the first open-sourced instruction-following LLMs for FMD task based on fine-tuning Llama3.1 with instruction data, the first multi-task FMD instruction dataset (FMDID) to support LLM instruction tuning, and a comprehensive FMD evaluation benchmark (FMD-B) with classification and explanation generation tasks to test the FMD ability of LLMs. We compare our models with a variety of LLMs on FMD-B, where our model outperforms other open-sourced LLMs as well as OpenAI's products. This project is available at https://github.com/lzw108/FMD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。