用rLoRA优化Qwen3-8B,提升金融文本分类效率与准确率。
Financial Text Classification Based On rLoRA Finetuning On Qwen3-8B model
- 结合噪声嵌入与rLoRA,实现高效低内存微调。
- 在新闻分类与情感分析任务中均超越多个主流模型。
- 适合实时金融NLP系统,兼顾性能与成本。
金融文本分类在量化交易系统中日益重要,包括金融情绪分析和新闻分类。本文评估了大语言模型Qwen3-8B在这两项任务上的表现。该模型具备强指令遵循与多语言能力,尤其针对高效微调和推理基准优化,适用于金融场景。我们采用基于前期工作的噪声嵌入指令微调方法,在监督适配中向嵌入层注入可控噪声以增强鲁棒性;进一步结合秩稳定低秩适应(rLoRA)与FlashAttention,显著降低训练内存占用并加快速度。在多项任务上,对比T5、BERT、RoBERTa等经典Transformer模型,以及LLaMA1-7B、LLaMA2-7B、Baichuan2-7B等大规模模型,Qwen3-8B均表现出更高分类准确率,且所需训练轮次更少。指令微调与内存高效优化的协同表明,Qwen3-8B可作为实时金融自然语言处理应用的可扩展、低成本解决方案,为未来动态量化交易系统提供有力基础。
原文摘要 · Abstract (English)
Financial text classification has increasingly become an important aspect in quantitative trading systems and related tasks, such as financial sentiment analysis and the classification of financial news. In this paper, we assess the performance of the large language model Qwen3-8B on both tasks. Qwen3-8B is a state-of-the-art model that exhibits strong instruction-following and multilingual capabilities, and is distinct from standard models, primarily because it is specifically optimized for efficient fine tuning and high performance on reasoning-based benchmarks, making it suitable for financial applications. To adapt this model, we apply Noisy Embedding Instruction Finetuning and based on our previous work, this method increases robustness by injecting controlled noise into the embedding layers during supervised adaptation. We improve efficiency further with Rank-stabilized Low-Rank Adaptation low-rank optimization approach, and FlashAttention, which allow for faster training with lower GPU memory. For both tasks, we benchmark Qwen3-8B against standard classical transformer models, such as T5, BERT, and RoBERTa, and large models at scale, such as LLaMA1-7B, LLaMA2-7B, and Baichuan2-7B. The findings reveal that Qwen3-8B consistently surpasses these baselines by obtaining better classification accuracy and needing fewer training epochs. The synergy of instruction-based fine-tuning and memory-efficient optimization methods suggests Qwen3-8B can potentially serve as a scalable, economical option for real-time financial NLP applications. Qwen3-8B provides a very promising base for advancing dynamic quantitative trading systems in the future.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。